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sequence  for  each  task  was  followed  and  each  new  topic  was  introduced  as 
needed.  This  provided  a logically-structured,  integrated,  functional, 
product-oriented,  learner-centered  approach.  Maximum  use  of  job  aids  and 
procedures  guides  were  used  in  carrying  out  the  Instructional  System  De- 
velopment (ISD^  process.  

. 

At  the  US  Army  Field  Artillery  School  a total  of  490  hours  were  required  for 
four  course  development  personnel  to  learn  to  convert  TEC  audio  visual 
lessons  to  computer  assisted  instruction  (CAI)  lessons  with  the  same  train- 
ing objectives.  Two  Observed  Fire  TEC  lessons  were  converted  to  PLANIT  CAI 
following  the  Instructional  System  Development  process  (ISD)  . These  lessons 
were  reviewed  by  subject  patter  experts,  and  a preliminary  evaluation  on 
Army  students  was  conducted.  Additionally,  four  other  TEC  lessons  were 
converted  to  CAI.  Average  cost  of  developing  each  of  four  CAI  lessons  with 
pre-  and  post  tests  was  128  man  hours  and  $681  in  hardware/ software  cost 
for  each  CAI  lesson. 

In  addition  to  the  six  lessons  converted,  contractor  project  personnel  de- 
veloped the  same  six  Observed  Fire  lessons  in  a Text  Presentation  (paper- 
and-pencil)  format  with  the  same  objectives  as  the  TEC  and  CAI  lessons. 
Module  pre-and  post  tests  for  on-line  computer  administration  for  the  six 
lessons  were  also  prepared.  These  efforts  provide  the  Field  Artillery 
School  with  three  media  for  six  Observed  Fire  lessons,  audio  visual,  CAI, 
and  paper-and-pencil  combinations  of  these. 

This  report  demonstrated  capability  to  develop,  execute,  evaluate  and  val- 
idate CAI  lessons. 

The  results  of  this  report  suggest  that:  CAI  can  be  cost  effective;  de- 
velopment and  evaluation  lead  time  can  be  short;  Army  lesson  developers 
can  be  trained  in  a relatively  brief  period.  Hence,  CAI  has  the  potential 
for  cost  savings  in  course  development,  evaluation,  and  administration. 

The  results  also  suggest  potential  training  effectiveness  as  a result  of 
individualized  self-paced  instruction  inherent  in  the  use  of  CAI,  and 
evaluation  capabilities  useful  in  the  management  of  the  instructional 
process . 
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Section  1.  EXECUTIVE  SUMMARY 


A.  PROBLEM 

Certain  instructional  situations  or  settings  may  permit  making  alternative 
delivery  systems  available  for  assignment  to  or  selection  by  trainees.  How- 
ever, there  is  little  sense  in  considering  alternative  delivery  systems  for 
the  same  training  objectives  unless  there  is  evidence  of  the  feasibility 
and  utility  of  their  employment  in  Army  training,  and  training  development 
settings.  Within  this  context,  the  following  questions  were  posed:  Does 
CAI  provide  a suitable  and  acceptable  media  for  delivering  TEC  materials  to 
field  units?  Can  Army  lesson  developers  feasibly  be  trained  to  convert  self- 
paced,  audio-visual  materials  into  CAI  format  and  easily  update  such  materials? 

B.  METHODOLOGY 

A three-week  training  workshop  was  conducted  to  train  Army  lesson  developers 
in  the  process  of  developing,  executing,  and  evaluating  CAI  courseware 
materials.  TEC  lessons  were  converted  in  this  process. 


The  workshop  was  followed  by  a six-week  conversion  course  in  which  lesson 
developers  converted  additional  TEC  materials  into  CAI  format,  executing  and 
evaluating  the  course  materials. 

The  approach  used  in  the  workshop  and  conversion  course  was  to  follow  the  job 
sequence  for  the  task  and  introduce  each  new  topic  as  it  was  needed 
in  this  process.  This  provided  a logically  structured,  integrated, 
product  oriented,  learner  centered  approach  to  the  workshop  and  the 
course.  Maximum  use  of  job  aids  and  procedures  guides  were  used  in 
out  the  Instructional  System  Development  process. 


and  applied 
functional, 
conversion 
carrying 
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C.  RESULTS 


Four  Directorate  of  Course  Development  personnel  at  the  U.S.  Army  Field  Artillery 
School  each  averaged  123  hours  of  workshop  training,  a total  of  490  hours,  to 
become  proficient  in  converting  TEC  audio  visual  lessons  to  computer  assisted 
instruction  (CAI)  lessons  with  the  same  training  objectives.  During  the 
workshop  the  four  course  developers,  in  teams  of  two,  following  the  Instructional 
System  Development  (ISD)  process,  converted  two  Observed  Fire  TEC  lessons  to 
PLANIT  CAI,  had  them  reviewed  by  subject  matter  experts,  and  conducted  a pre- 
liminary evaluation  on  Army  students. 

During  the  six-week  conversion  course,  each  of  the  four  personnel  converted  an 
additional  TEC  lesson  on  Observed  Fire,  completing  the  series  of  six  lessons 
to  CAI.  These  were  also  reviewed  by  subject  matter  experts  for  technical 
accuracy  and  current  tactical  doctrine  and  a preliminary  evaluation  conducted 
using  Army  students.  Average  cost  of  developing  each  of  the  four  CAI  lessons 
with  pre-  and  post  tests,  approximately  lh  hours,  through  preliminary 
evaluation  (small  group  tryout)  is  as  follows: 

Course  Developer  - 111  hours 

Subject  Matter  Expert 

(2  @ l*a  hours)  - 3 hours 

Students  (tryout  - 7 @ 2 hrs.)  - 14  hours 

Computer  Costs  - $381 

Telephone  Costs  - $300 

The  above  totals  to  128  man  hours  and  $681  for  each  CAI  lesson. 

The  course  developers  prepared  their  own  off-line  course  exhibits.  Course 
listings,  test  scoring,  student  records,  and  other  course  development  adminis- 
tration was  done  by  the  computer  and  included  in  the  above  costs.  Calendar 
time  for  development  of  a lesson  through  preliminary  evaluation  is  30  working 


Subject  matter  expert  and  student  reaction  to  the  six  Observed  Fire  CAI 
lessons  was  excellent. 

In  addition,  project  personnel  have  developed  the  same  six  Observed  Fire  lessons 
in  a Text  Presentation  (paper-and-pencil)  format  with  the  same  objectives  as 
the  TEC  and  CAI  lessons.  Module  pre-  and  post  tests  for  on-line  computer 
administration  for  the  six  lessons  have  also  been  prepared.  This  provides  the 
Field  Artillery  school  with  three  media  for  the  six  Observed  Fire  lessons, 
audio  visual,  CAI,  and  paper-and-pencil  and  combinations  of  these. 

The  U.S.  Army  Field  Artillery  School  personnel  involved  are  capable  of  develop- 
ing their  own  CAI  lessons,  executing  them,  evaluating  and  validating  them  via 
computer.  The  school  is  currently  in  process  of  conducting  a large  group 
validation  of  the  Observed  Fire  CAI  lessons  and  will  be  updating  changes  in 
tactical  doctrine  for  40  hours  of  PLANIT  courseware  which  exists  in  the  USAFAS 
TACFIRE  Department.  Other  areas  for  CAI  courseware  development  are  being 
considered . 

D.  IMPLICATIONS  FOR  ARMY  TRAINING 

The  results  show  that  CAI  development  costs  are  low,  the  development  and 
evaluation  lead  time  is  short,  and  Army  lesson  developers  can  be  trained  in 
a short  time  period.  The  CAI  system  provides  cost  savings  in  computerizing 
the  course  development,  evaluation,  and  administration  process.  Lessons  are 
listed,  changes  (editing)  are  made,  lessons  are  executed,  tests  are  scored, 
and  prescr iptives  generated  by  computer  action.  At  the  same  time,  CAI  fully 
incorporates  a more  truly  individualized  self-paced  mode  of  course  presenta- 
tion with  computerized  control  of  the  learning  process.  CAI  has  a further 
benefit  that  early  in  the  ISD  process,  course  material  is  in  the  format  as 
the  student  sees  it.  This  facilitates  the  subject  matter  expert  review  and 
course  execution  process. 
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CAI  materials  are  quickly,  easily,  and  inexpensively  updated  as  changes  in 
tactical  doctrine  occur. 

Further,  the  requirement  in  CAI  that  the  student  gets  feedback  and  advances 
only  after  committing  to  a substantive  response  suggests  that  the  CAI  process 
requires  closer  adherence  to  the  ISD  process  In  that  the  student  either  meets 
the  objectives  or  does  not.  Data  backing  the  ISD  process,  e.g.,  course 
listings,  student  lesson  interaction,  and  lesson  and  module  tests,  are  easily 
and  automatically  produced  and  scored  by  the  computer.  These  records  now 
provide  the  evaluation  and  validation  data,  including  attitudinal  data,  which 
ensures  that  the  ISD  process  and  standards  have  been  met.  CAI  has  the  benefit 
of  clarifying  and  simplifying  the  ISD  process  to  course  developers,  in  that 
use  of  the  CAI  authoring  language  forces  a structured,  highly  procedural 
task  centered  approach. 


Section  2:  INTRODUCTION 


A.  PROJECT  BACKGROUND 

In  the  current  TEC  Media  research  effort,  sponsored  by  the  U.S.  Army  Field 
Artillery  School  (USAFAS) , the  U.S.  Army  Research  Institute  (ARI)  addresses 
several  interrelated,  continuing  needs:  (1)  To  be  able  to  rapidly  up-date 
field  manuals  and  other  training  materials,  and  (2)  To  make  these  materials 
available  to  men  in  units  who  must  maintain  or  improve  their  Military  Occupa- 
tional Speciality  (MOS)  proficiency,  and  (3)  Above  all,  to  ensure  that 
training  be  conducted  with  maximum  efficiency,  effectiveness,  and  economy. 

Traditional  methods  of  instruction  have  not  been  entirely  successful  in  ful- 
filling these  needs,  even  though  substantial  time,  energy,  and  money  have 
been  expended  in  preparing  field  manuals  and  other  training  materials.  Not 
only  is  such  training  literature  frequently  unavailable  to  men  in  units,  but 
often  what  is  available  is  out  of  date  with  respect  to  changes  in  doctrine 
and/or  weaponry. 

In  1971,  by  direction  of  the  Chief  of  Staff,  U.S.  Army,  the  Board  for  Dynamic 
Training  was  established  and  tasked  with  recommending  ways  "To  make  training 
in  units  more  exciting  and  meaningful."  Among  their  key  recommendations  was 
a suggestion  to:  "Provide  MOS-related  unit  training  extension  courses  (UTEC) 
employing  multi-media  materials  applicable  for  both  individual  and  small  group 
study." 

Pursuant  to  this  suggestion,  the  U.S.  Army  Combat  Training  Board  (which  was 
established  as  a follow-up  to  the  Board  for  Dynamic  Training)  established  a 
program  to  develop  training  extension  course  (TEC)  material.  Briefly,  TEC 
materials  consist  of  self-paced  visual,  audio,  or  audio-visual  instructional 
packages  dealing  with  subjects  which  are  common  to  the  four  ccnbat  arms 
branches  of  the  Army  (infantry,  Armor,  Field  Artillery,  and  Air  Defense).  TEC 
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lessons  for  various  military  occupational  specialties  (MOS)  have  been  developed 
and  are  currently  undergoing  field  testing,  evaluation,  and  revision. 

These  new  materials  are  expected  to  constitute  an  improvement  over  traditional 
instruction,  particularly  from  the  point  of  view  of  effectiveness  and 
efficiency.  Nevertheless,  what  problems  will  be  encountered  with  respect  to 
rapid  updating  remain,  as  yet,  undetermined.  It  is,  in  part,  this  demand  for 
rapid  update  capability  which  provides  a major  impetus  for  exploring  alternative 
delivery  systems.  Since  one  advantage  of  computers  is  their  facilitative  effect 
on  authoring,  editing,  and  updating  of  course  content,  CAI  was  the  vehicle 
chosen  for  the  first  phase  of  the  present  study,  with  the  possibility  of 
additional  media  to  be  considered  in  later  research. 

Although  many  CAI  efforts  have  relied  upon  dedicated  systems,  all  indications 
suggest  that  time-shared,  mini,  and  tactical  computers  may  also  be  deployed 
effectively.  Indeed,  the  Army,  which  has  already  successfully  developed  and 
fielded  several  tactical  data  systems,  has  begun  to  look  into  this  question. 
Because  it  is  likely  that  the  conduct  of  peace-time  tactical  operations  will 
not  fully  utilize  the  data  processing  capability  of  these  computers,  studies 
have  been  undertaken  to  determine  the  expedience  of  supporting  unit  and 
individual  training  requirements  with  these  systems  when  they  are  not  engaged 
in  tactical  operations.  The  Army  Research  Institute  (ARI)  demonstrated  the 
feasibility  of  this  concept  in  the  Development  Tactical  Operations  Systems 
(DEVTOS)  located  at  the  Modern  Army  Selected  Systems  Test  Evaluation  and 
Review  (MASSTER)  facility  at  Fort  Hood,  Texas. 

As  part  of  this  ontinuing  research  effort,  ARI  has  compared  the  estimated  cost 
data  of  TEC  training  with  conventional  Army  classroom  instruction  and  has  also 
compared  the  effectiveness  of  TEC  training  with  that  of  conventional  Army 
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classropm  instruction.  The  results  of  this  research  have  been  published  in 
the  following  ARI  research  reports. 

Research  Problem  Review  75-3,  A Cost  Assessment  of  Army  Training 
Alternatives,  August  1973 

Research  Report  1188,  Training  Individuals  in  Army  Units:  Compara- 
tive Effectiveness  of  Selected  TEC  Lessons  and  Conventional  Methods, 
December  1975 

B.  PURPOSE  OF  THE  PROJECT 

While  the  overall  purpose  of  the  TEC  Media  research  program  was  to  determine 
the  efficiency  and  effectiveness  of  alternatives  for  providing  TEC  materials 
to  units  in  the  field,  the  present  effort  addresses  one  sophisticated  alter- 
native delivery  system  for  TEC;  namely.  Computer  Assisted  Instruction.  This 
study  examined  the  potential  advantages  in  implementing  CAI  in  training 
extension  courses.  The  major  thrust  was  not  to  pit  CAI  directly  against  other 
training  media.  Implicit  in  such  a treatment  would  be  the  assumption  that 
alternative  delivery  systems  are  competitive  rather  than  complementary.  As  a 
planning  concept,  TEC  is  not  wedded  to  any  particular  device,  medium,  or 
technique.  Indeed,  there  has  been  consistent  Army-wide  support  (throughout 
both  the  training  and  research  establishments)  for  exploring  a full  range  of 
alternative  delivery  systems  for  TEC.  A study  of  the  TEC  to  CAI  conversion 
process,  as  operable  within  an  Army  setting,  represents  the  initial  phase  of 
this  type  of  research.  The  current  study  investigated  the  requirements, 
associated  procedures,  resources,  materials,  and  costs  necessary  to  convert 
TEC  lessons  to  CAI  lessons  which  teach  to  the  same  training  objectives. 

The  immediate  goal  was  to  ascertain  the  feasibility  of  having  Army  (in  this 
case.  Field  Artillery  School)  lesson  developers  convert  self-paced,  audio- 
visual materials  into  CAI  format. 
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C.  PROJECT  OBJECTIVES 


The  objectives  of  this  effort  were  to: 

1.  Provide  a formative  (preliminary)  evaluation  of  the  suitability  of  CAI 
as  one  means  for  providing  TEC  instruction. 

2.  Determine  the  amount  of  time  involved  and  the  types  of  difficulty 
encountered  in  having  Field  Artillery  School  lesson  writers  convert 
audio-visual  TEC  material  to  appropriate  CAI  format. 

3.  Provide  documentation  of  CAI-to-TEC  conversion  costs  and  procedures  for 
conducting  a more  formalized  training  effectiveness  evaluation. 

4.  Determine  the  training  time  required  to  provide  Army  lesson  developers 
with  the  capability  to  produce  CAI  versions  of  TEC. 

5.  Make  available  three  alternative  versions  of  six  selected  TEC  lessons 
(i.e.,  CAI  stand-alone,  CAI/Audio-visual  Combination,  Text  Presentation). 

6.  Provide  the  USAFAS  with  a course  development  cadre  trained  in  the  practical 
steps  of  converting  self-paced  TEC  audio  visual  lessons  to  CAI,  and  in 

the  use  of  the  computer  system  to  input,  check,  edit,  and  update  CAI 
materials. 

The  scope  of  the  project  in  conformance  with  the  above  objectives  are  shown  in 

Appendix  A. 

D.  ORGANIZATION  OF  THIS  FINAL  REPORT 

Information  In  this  final  report  follows  the  sequence  in  which  the  tasks  were 

accomplished  as  follows: 

• Section  1 - Executive  Summary 

• Section  2 - provides  the  background,  purpose,  objectives,  and  scope 
of  the  project. 
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• Section  4 - explains  how  the  workshop  and  conversion  course  were 
conducted 

• Section  5 - provides  the  results  obtained. 

• Section  6 - explains  the  development  of  text  and  the  modular  pre- 
and  post  tests  used  with  the  three  alternative  versions  of  the  six 
lessons  on  Observed  Fire  (TEC,  CAI,  TEXT). 

• Section  7 - states  the  conclusions  and  recommendations  stemming  from 
the  project. 

The  Appendix  contains  many  of  the  procedures  guides,  job  aids,  experience 
questionnaires,  student  records,  lesson  listings,  and  attitude  questionnaires 
developed  and  used  in  this  project.  Use  of  these  materials  is  explained  in 
the  text  for  the  appropriate  section. 
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Section  3.  PREPARATIONS  FOR  THE  TECMEDIA  CAT 
WORKSHOP  AND  CONVERSION  COURSE 


The  workshop  preparations  involved  four  major  areas  as  follows: 

A.  Defining  System  Parameters 

B.  TEC-CAI  Conversion  Requirements 

C.  TEC-CAI  Conversion  Data  Collection  System 

D.  TEC-to-CAI  Conversion  Workshop  Plan 

The  procedures,  products,  requirements,  and  assumptions  for  each  of  these  are 
covered  in  the  paragraphs  which  follow. 

A.  DEFINING  SYSTEM  PARAMETERS 

One  of  the  first  activities  in  this  type  of  project  is  to  define  what  systems 
are  being  used  and  the  parameters  and  limitations  of  each.  These  in  turn 
determine  the  preparations  required  to  train  individuals  in  the  operation  and 
effective  use  of  the  systems.  In  this  project,  the  following  systems  came 
into  use: 

• UNIVAC  1108  computer  and  EXEC  8 operating  system 

• ADDS  Consul,  with  telephone  modem,  and  slave  printer 

• UNIVAC  Data  Base  Management  System  and  Text  Editor 

• PLANIT  CAI  Language  and  System 

Three  instructional  techniques  were  employed  to  describe  the  operation  and  use 
of  these  systems. 

• Classroom  instruction  incorporating  "chalk  talk"  explanations. 

• Reference  and  use  of  a specially  prepared  procedures  guide  containing 
detailed  procedural  steps,  results  of  action,  and  comments 
(stimulus,  action,  response). 


3-1 


I 


• "Hands-on"  familiarization  and  use  of  each  system  via  the  ADDS 
880  consul. 

Instruction  in  accessing  the  UNIVAC  1108/EXEC  8 Operating  System  included  steps 
to  "power  up"  the  peripheral  equipment,  establish  data  phone  communication  with 
the  computer  located  at  Edgewood  Arsenal,  and  for  entering  the  job  statements 
correctly  and  in  the  proper  sequence  at  the  ADDS  terminal  to  initiate  inter- 
action with  the  desired  program  or  operation  (e.g..  Text  Editor  or  PLANIT) . 

(See  Appendix  B,  Figure  B-l.) 

Course  participants  use  of  the  Text  Editor  system  was  a means  of  building  a 
card  image  file  that  would  subsequently  serve  as  an  input  to  the  CAI  PLANIT 
system. 

Course  participants  were  given  appropriate  instruction  in  how  to  access  the 
Text  Editor  and  the  capabilities  and  differences  between  the  two  basic 
operating  modes — input  (Appendix  B,  Figure  B-2)  and  edit  (Appendix  B, 

Figure  B-3).  A subset  of  editing  commands  by  which  students  could  modify  the 
card  image  input  file  was  Included  in  the  procedures  guide  for  student  use. 
Appropriate  instruction  and  on-line  practice  were  provided.  The  procedures 
guide  reference  document  was  consulted  as  required  as  an  aid  to  instruction 
and  use  of  the  Text  Editor. 

Before  using  the  PLANIT  system,  the  course  participants  received  intensive 
off-line  instruction  as  to  the  system's  general  capabilities  and  requirements. 
Language  syntax  and  conventions  that  dictated  the  format  in  which  TEC-to-CAI 
converted  material  was  to  be  developed  were  presented  and  reviewed  in  detail. 
On-line  instructions  (Appendix  B,  Figure  B-4)  Included  procedures  to  "prestore" 
and  "build"  the  lesson  material  from  the  card  input  files  that  were  created 
using  the  Text  Editor,  how  to  execute  and  checkout  the  lesson  material,  and 
use  of  the  editing  features  of  PLANIT  to  make  modifications 
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(Appendix  B,  Figure  B-5)  prior  to  validating  the  lesson  material  using  Army 
personnel.  "Prestore"  and  "Build"  are  PLANIT  commands  to  enter  lessons  from 
card  image  format  into  the  PLANIT  System  for  execution.  Course  participants 
also  received  instruction  on  PLANIT  record  keeping  capabilities  and  how  to 
access  and  retrieve  student  performance  records.  As  a final  step,  instruction 
and  procedures  were  given  on  how  to  make  the  developed  lessons  "permanent"  by 
transferring  these  lessons  from  the  PLANIT  system  to  permanent  storage  using 
the  Data  Base  Management  System  within  the  UNIVAC  1108 . The  procedures  guide 
reference  document  again  served  as  an  adjunct  instructional  aid  to  the  on-line 
and  off-line  instruction  concerning  the  use  of  the  PLANIT  system. 

B.  TEC-CAI  CONVERSION  REQUIREMENTS 

The  TEC-to-CAI  conversion  requirements  specify  the  procedures,  resources,  and 
materials  which  are  required  to  convert  TEC  lessons  to  CAI  lessons  which  teach 
to  the  same  training  objectives,  yet  which  enhance  the  reinforcement,  test, 
and  critical  capabilities  unique  to  the  CAI  training  mode. 

The  conversion  requirements  served  to: 

• provide  performance-oriented  objectives  and  sequence  for  the 
three-week  TECMEDIA  CAI  Workshop. 

• specify  the  tasks  for  which  time  and  difficulties  data  were 
collected  during  the  preparation  of  CAI/TEC  materials  by  Army 
personnel. 

• specify  the  procedure  which  determined  the  design  and  operation 
of  the  TEC/CAI  conversion  data  collection  system. 

• provide  a procedure  for  converting  TEC  audio  visual  materials  to 
CAI  materials  which  is  operable  within  an  Army  setting. 

• provide  a procedure  which  is  independent  of  lesson  content  or  type 
of  test. 
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The  four  conversion  steps  for  preparing  TEC  materials  for  CAI  are: 

1.  Review  TEC  Lesson  with  TEC  Script 

2.  Modify  Annotated  TEC  Script 

3.  Rewrite  Script  in  Frame  Presentation  Order 

4.  Encode  Frame  Sequence  for  CAI  (PLANIT) 

The  workbook  "TEC-to-CAI  Examples”  was  prepared  as  a procedures  guide  showing 
the  inputs  and  procedures,  with  two  completely  worked  examples  using  the  LAW 
TEC  lesson  training  objectives.  The  workbook  provided  a working  reference 
for  each  course  designer  during  the  workshop  and  conversion  course.  The 
approach  used  was  to  have  easily  managed,  clearly  defined,  easily  learned 
step-by-step  procedures  which  introduced  new  material  at  the  time  it  was 
needed  and  could  be  applied.  Sample  pages  of  the  workbook  showing  the  four 
steps  are  contained  in  Appendix  C.  These  steps  were  incorporated  into  the  five 
major  conversion  tasks,  shown  in  Table  3-1. 

A brief  synopsis  of  each  of  these  tasks  is  as  follows: 

Task-1.  Review  a CAI  Lesson.  The  purpose  of  this  task  is  to  provide  an  over- 
view of  the  CAI  end  product,  i.e.,  the  organization,  frame  formats,  and 
execution  order  of  a representative  CAI  lesson  (e.g.,  a LAW  lesson).  The 
Army  training  specialist  reviews  a CAI  lesson  listing  and  student  handouts, 
while  executing  the  lesson  at  a terminal,  noting  how  the  frame  constructions 
in  the  listing  control  displays,  response-matching,  feedback,  and  lesson 
execution  order.  He  obtains  portions  of  the  execution  record  at  the  terminal 
printer  to  verify  how  lesson  decision  frames  operate  to  provide  conditional 
feedback  and  branching  according  to  performance  standards  for  the  lesson  pre- 
test, lesson  segment,  or  end  of  lesson  test.  In  the  training  workshop,  this 
task  also  serves  as  a vehicle  for  teaching  points  on  the  capabilities, 
strengths,  and  restrictions  of  the  particular  CAI  training  mode. 
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Table  3-1.  TEC-to-CAI  Conversion  Tasks 


Review  a CAI  Lesson 

Obtain  a CAI  Lesson  Listing 
Execute  CAI  Leseoa  & Teats 
Compare  CAI  Leaaon  with  Llatlng 
Review  Execution  Records 


Review  TEC  Lesson  4 Materials 

Execute  TEC  Lesson  to  be  Converted 
Identify  Areas  of  Change  for  Conversion 
Gather  & Review  Related  Content  Materials 


Prepare  TEC  for  CAI  Conversion 


Modify  & Sequence  Existing  Display  Frames 
Modify  & Sequence  Existing  Question  Frames 
Prepare  & Sequence  New  Question  Frames 
Specify  Branching  for  Correct/Incorrect  Responses 
Prepare  End  of  Lesson  Test  ASpeclfy  Decisions 
Organize  Lesson  Pretest  & Specify  Decisions 


Prepare  Material  In  PLAHIT 

Review  PLANIT  Conventlona  & Constraints 
Encode  Lesson  & Test  Frames  In  PLANIT 
Input  Lesson  & Tests  at  Console 
Check  PLANIT  Listings  i Student  Handouts 
Checkout  & Edit  Materials  at  Console 


Evaluate  4 Revise  Materials 

Obtain  Listings  & Student  Handouts 

SME  Review  of  Materials 

Edit  Materials  Based  on  SME  Review 

Conduct  Student  Tryouts 

Edit  Materials  Baaed  on  Tryouts 

Reproduce  Student  Materials 


Available  or  use  of 
PLANIT  commands. 
PLANIT  conuads. 
Visual  Inspection. 
PLANIT  commands. 


Operate  Beseler  Cue-See. 
Annotate  TEC  script. 
Available  Army  material. 


Edit  script  & visuals. 
Change  for  CAI. 

Write  for  CAI. 

Update  frames. 

Add  new  frames. 

Add  new  fraaws. 


Checklist  & references . 
Notebook  deskwork. 
PLANIT  entries. 

PLANIT  & inspection. 
PLANIT,  student  handout 
& job-aid 


PLANIT  & reproduction. 
Inspect  listings  & handouts. 
PLANIT/graphlc  arts. 

PLANIT,  records  & Interview. 
PLANIT/graphlc  arts. 
Reproduction. 
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Task-2.  Review  TEC  Lesson  and  Materials.  In  this  task  the  Army  training 
specialist  reviews  the  objectives,  lesson  pretest,  lesson  test  items,  the 
structure  and  content  of  the  TEC  lesson  to  be  converted  to  CAI.  As  he  takes 
the  lesson  on  the  Beseler  Cue-See,  he  identifies  and  marks  in  the  TEC  audio 
script  (containing  visual  cues)  those  areas  of  the  TEC  lesson  that  need 
expansion  or  change  for  CAI — for  example:  TEC  questions  that  must  be  supple- 
mented with  anticipated  correct-incorrect  responses  ana  associated  feedback; 
portions  of  the  script  where  additional  questions  are  needed;  existing  TEC 
pretest  items  that  require  alternative  forms  for  the  CAI  lesson  post  test; 
and,  visuals  that  can  be  used  directly  or  with  modifications  in  the  CAI 
student  handouts.  He  also  determines  the  need  for,  gathers,  and  familiarizes 
himself  with  the  content  of  related  FMs,  TMs,  and  other  Army  materials.  This 
work  provides  a basis  for  the  next  major  task. 

Task-3.  Prepare  TEC  for  CAI  Conversion.  The  purpose  of  this  first-stage 
conversion  task  is  to  modify,  supplement,  and  organize  frames  of  the  existing 
TEC  lesson  so  as  to  be  directly  convertible  to  the  PLANIT  CAI  language  and 
adjunct  visual  '.uandouts.  An  existing  TEC  frame  sequence  consists  of  audio- 
visual information,  instruction,  questions,  and  comparison  material  for 
student  self-check  of  answers.  Much  of  the  information,  instructions,  and 
questions  will  be  transferable  to  on-line  CRT  displays.  What  is  not,  is 
transferred  to  off-line  printed  visuals  either  directly  or  with  minor  changes. 
Constructed  response  questions  in  TEC  will,  for  CAI,  require  that  correct 
and  incorrect  responses  be  anticipated  and  listed,  while  all  types  of  questions 
will  require  appropriate  feedback  statements  for  correct,  incorrect,  and 
unanticipated  student  responses.  TEC  sequences  may  need  additional  questions 
to  ensure  mastery  of  component  objectives  of  multiple-part  criterion  test 
items.  New  questions  written  for  the  CAI  mode  incorporate  appropriate  antici- 
pated answers  and  answer  feedback,  including  extra  (remedial)  frames  for 
incorrect  answers.  Performance  standards  and  conditional  feedback  or  lesson 
control  is  specified  at  pretest,  lesson  segment,  and  lesson  test  checkpoints 
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for  the  CAI  mode.  The  outcome  of  this  task  is  a numeric  sequence  of  "frames" 
and  draft  visuals  compatible  with  the  original  TEC  lesson  but  modified  and 
supplemented  for  direct  conversion  to  CAI.  -This  material  is  organized  and 
numbered  in  the  training  specialist's  conversion  notebook. 

Task-4.  Prepare  Material  in  PLANIT.  The  training  specialist  modifies  material 
in  his  lesson  notebook  from  Task-3  in  accordance  with  conventions  of  the 
PLANIT  CAI  language.  He  then  inputs  his  lesson  directly  to  the  computer  from 
the  terminal  keyboard  using  either  the  Text  Editor  system  or  the  author  mode 
of  PLANIT.  Entry  of  textual  presentation,  questions,  and  feedback  statements 
is  straightforward.  Special  language  features  are  used  to  specify  answer- 
matching, decision  criteria,  recordkeeping,  and  control  requirements  of  the 
CAI  lesson.  Upon  entering  his  lesson  frames,  the  author  obtains  a lesson 
listing  at  the  terminal  printer  and  then  self-administers  the  lesson  frames 
in  the  student  mode  of  PLANIT  to  check  that  they  execute  as  intended  and  with- 
out error  halts.  Where  errors  are  found,  he  uses  PLANIT  commands  to  edit 
the  lesson  in  accordance  with  guidance  provided  by  the  PLANIT  errors  checklist. 
His  on-line  check  includes  the  draft  copies  of  the  printed  student  adjunct 
materials  so  as  to  ensure  compatibility  of  the  CAI  lesson  with  off-line 
visuals  and  clarity  of  usage  instructions. 

Task-5.  Evaluate  and  Revise  Materials.  This  final  task  assures  that  the 
converted  CAI  materials  are  suitable  for  initial  student  use.  Army  subject 
matter  experts  review  the  CAI  lesson  listings  and  draft  visuals  for  appro- 
priateness of  technical-procedural  content  and  tactical  doctrine,  noting 
changes  in  the  lesson  listing  and  providing  clarification  to  the  training 
specialist.  The  specialist  then  edits  his  CAI  lesson  and  visuals  in  accordance 
with  the  SME's  findings,  and  sufficient  copies  of  the  visuals  are  made  for 
testing  the  lesson  with  a small  number  of  students.  This  tryout  with  Army 
students  naive  to  the  subject  matter  and  representative  of  the  ability  range 
of  the  target  student  group  serves  three  purposes:  (1)  to  improve 
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language-level,  clarity,  and  adequacy  of  remedial  material  in  the  lessons  in 
accordance  with  needs  expressed  by  students  while  taking  each  lesson;  (2)  to 
ensure  that  the  CAI  instructions  .and  procedures  are  appropriate;  and  (3) 
provide  a small-sample  verification  that  students  meet  lesson  test  standards 
after  CAI  training.  Revisions  to  materials  are  made  in  accordance  with  findings 
of  the  tryout.  Final  lesson  listings  and  off-line  exhibits  are  then  produced 
for  the  field  studies. 

The  five  conversion  tasks  given  above  are  further  detailed  in  Table  3-2, 
"TEC-to-CAI  Conversion  Requirements."  This  table  shows  the  Input  (support) 
Requirements,  the  Activity,  and  the  Output/Product  (task  outcomes).  These 
procedures  follow  the  TRADOC  ISD  model  as  applied  to  TEC  materials,  given  that 
the  training  objectives  and  tasks  have  already  been  specified. 

C.  TEC-CAI  CONVERSION  DATA  COLLECTION  SYSTEM 

The  TEC-to-CAI  Conversion  requirements  provided  the  basis  for  the  data 
collection  system.  This  system  was  designed  to  obtain  data  on  time  and 
difficulties  and  other  factors  relevant  to  the  purpose  and  conduct  of  the  work- 
shop and  conversion  course.  The  forms  and  logs  developed  and  their  purpose 
are  as  follows:  Copies  of  the  forms  are  included  in  Appendix  D. 

1.  TEC-CAI  Background  Data  Questionnaire  - used  to  provide  experience  data 
of  the  course  participants.  Completed  on  the  first  day. 

2.  Daily  Individual  Activities  Log  - provides  a daily  self-report  of  the 
activities  of  each  of  the  course  participants.  Collected  daily  during 
the  workshop  and  conversion  course. 

3.  Monitor's  Observation  Log  - used  by  SDC  personnel  to  provide  a daily 
observational  audit  of  task  time,  difficulties,  and  corrective  actions 
occurring  each  day  during  the  workshop  and  conversion  course. 
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3.1.6  Draft  changes  to  copies  of  visuals.  Draft  changes  oi 

Notebook  3.1.7  Sequence  changed  presentations.  Lesson  Notebook 


TEC-to-CA.1  Conversion  Requirements  (Cont. 


3.5.1  Identify,  modify  & prepare  questions. 

3.5.2  Write  test  Instructions.  Instructions  & questions 
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Terminal  connect  to  4.3  Input  Lesson  & Tests  at  Console  CRT/hardcopy  displays 

PLANIT  & terminal  X 

printer. 


Table  3-2.  TEC-to-CAI  Conversion  Requirements  (Cont. 


4.  Questionnaire  (SME)  - provides  experience  data  on  the  Subject  Matter 
Expert  (SME).  Each  SME  completes  the  form  once. 

5.  CAI  Lesson  Review  - provides  data  on  the  technical  accuracy  of  each 
lesson  and  whether  the  objectives  of  the  CAI  and  TEC  lessons  are  the 
same.  Completed  by  the  SME  after  review  of  the  lesson. 

6.  Questionnaire  (S)  - provides  data  on  students  taking  the  lesson,  particu- 
larly in  terms  of  whether  the  prerequisites  for  the  course  (land  naviga- 
tion and  map  reading)  have  been  met.  Completed  by  the  student  prior 

to  taking  the  CAI  lessons. 

7.  CAI  Student  Attitude  Questionnaire  - provides  data  on  student  acceptance 
and  reaction  to  lesson  content  and  media.  Completed  by  the  student 
following  the  on-line  session. 

8.  Student  interaction  record  - printout  of  lesson  and  test  execution  by 
the  student  (frame  presentation  and  responses).  Obtained  by  turning  on 
the  printer  attached  to  the  console  prior  to  executing  the  lessons  and 
tests. 

9.  PLANIT  Student  Record  - computer  printout  of  student  performance  data  for 
each  lesson  and  test.  Automatically  recorded,  scored,  and  stored  during 
execution.  Obtained  after  students  have  completed  lessons  and  tests. 

The  above  records,  forms,  and  logs  provide  a data  collection  system  which 
permits  accurate  records  to  be  kept  with  respect  to  length  of  time  spent, 
difficulties  encountered,  and  acceptability  of  the  product.  The  system  is 
applicable  to  any  TEC/CAI  subject  matter  conversion.  Much  of  the  system  can 
also  be  used  in  developing  any  course  materials  using  the  Instructional 
System  Development  (ISD)  process. 
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D.  TEC-TO-CAI  CONVERSION  WORKSHOP  PLAN 


The  approach  used  in  developing  the  model  for  the  workshop  was  to  follow  the 
developed  job  sequence  for  the  conversion  tasks,  and  to  introduce  each  new 
topic  as  it  was  needed  and  applied  in  this  process.  Further,  only  those 
elements  of  a topic  that  would  be  used  would  be  presented  at  that  particular 
time,  but,  by  the  time  the  workshop  was  over,  all  required  elements  and  topics 
would  have  been  applied. 


This  provided  a logically  structured,  integrated,  functional,  product  oriented, 
learner  centered  approach  to  the  workshop  and  the  conversion  course. 


The  TEC-to-CAI  Conversion  Workshop  Schedule  is  given  in  Appendix  E.  The  work- 
shop model  is  organized  into  three  training  blocks  in  accordance  with  the 
primary  product  outcomes  of  each  conversion  activity.  These  are: 


Conversion  Activity 
Prepare  TEC  Material  for  CAI 
Prepare  Material  in  PLANIT 
Evaluate  and  Revise  Lesson 


Converted  Product 
Lesson  Conversion  Notebook 
Lesson  Executable  on  PLANIT 
Lesson  Effective  and  Acceptable 


This  particular  plan  assumes  that  two  CAI  lessons  were  to  be  converted  during 
the  three-week  workshop  by  the  four  Army  course  development  personnel  assigned 
to  the  workshop.  It  assumes  also  that  four  more  TEC  lessons  were  to  be  con- 
verted during  the  subsequent  six-week  conversion  period. 


After  the  initial  setup  and  first  workshop  day,  the  emphasis  is  increasingly 
on  performing  conversion  tasks  with  only  that  on-the-job  training  needed  to 
move  from  task-to-task  and  step-to-step . 

A brief  synopsis  of  each  training  block  follows: 
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Block  1 - Prepare  TEC  Materials  for  CAI  Conversion 


Product  Objective.  Army  lesson  writers  prepare  Lesson  Conversion  Workbook,  ready 
for  encoding  in  PLANIT,  from  TEC  materials  selected,  modified,  and  supplemented 
in  accordance  with  CAI  design  requirements. 

Training  Topic 


Workshop  Preview 

Beseler  Cue-See  Familiarity 

Team  Exercise  on  Cue-See 

Use  of  TEC  Materials  and  Lesson 
Conversion  Notebook 

Overview  of  CAI  and  PLANIT 

Use  of  CAI  Console  (Exercise) 

Overview  of  a Typical  CAI  Module 

TEC-to-CAI  Design  Requirements 

Frame-Thinking,  Part  I:  Presenting 
Information  and  Questions 

Review  Example  Conversion  Notebook 

Frame-Thinking,  Part  II:  Writing 
Decisions.  Decision-Writing 
Practice 

CAI  Considerations 
TEC/CAI  Testing  Requirements 


Conversion  Task 


Review  a CAI  Lesson 

Review  TEC  Lesson  and  Materials 
Prepare  TEC  for  Conversion 


Deskcheck  Conversion  Notebook 


Block  2 - Prepare  Lesson  Materials  in  PLANIT 

Product  Objective.  Army  lesson  writers  encode  material  in  their  Lesson  Conversion 
Notebook  in  PLANIT,  enter  it  into  the  computer,  and  determine  that  the  lesson 
operates  without  errors. 
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Product  Standards.  This  checkpoint  is  to  ensure  that  all  lesson  material 
has  been  entered  into  the  computer  and  that  it  executes  in  the  student  mode 
of  PLANIT  without  halt,  as  intended: 

• All  lesson  frames  entered  into  the  computer. 


• Lesson  executes  without  error  messages  or  halts. 


• Lesson  executes  as  intended 
Training  Topic 

PLANIT  Conventions  and  Legalities, 
Parts  I and  II,  M & Q Frames 

PLANIT  Conventions  and  Legalities, 
Part  III,  D-Frames 

Entering  Frames  to  the  Computer 


Executing,  Checking,  and  Editing 

with  PLANIT 
* 


by  the  lesson-writer. 

Conversion  Task 

Encode  (M)ultiple-choice  and 
(Q)uestion  frames 

Encode  (D)ecision  frames 

Enter  frames  at  console 
(Team  1 and  Team  2) 

Deskcheck  materials 

On-line  Check  and  Editing  of 
converted  lesson 
(Team  1 and  Team  2) 


Block  3 - Evaluate  and  Revise  Converted  Materials 

Product  Objective.  Update  lesson  to  include  changes  from  Subject  Matter 
Expert  (SME)  review  and  lesson  tryouts  with  naive  students  to  indicate  perfor- 
mance, time,  and  acceptance  standards  are  met. 

Product  Standards.  This  final  checkpoint  is  to  ensure  lesson  effectiveness 
and  acceptance: 

• SME  review  comments  incorporated. 

• Students  perform  to  standard  on  post-test. 

• Student  acceptance  medium- to-high. 


Training  Topic  Conversion  Task 

Prepare  for  SME  Review 

Conduct  SME  Review 
(Team  1 and  Team  2) 

Edit  Materials  Based  on  SME  Review 
(Teams  1 and  2 stagger  schedule) 


Conduct  Student  Tryouts 

(Teams  1 and  2 stagger  schedule) 

Compile  Data  and  Determine  Revisions 
(Team  1 and  Team  2) 


Make  Final  Revisions 

(Teams  1 and  2,  as  required) 

A special  feature  of  the  workshop  plan  and  preparations  was  the  development 
of  outlines,  summaries,  guidelines,  and  other  job  aids  and  procedures  guides 
for  carrying  out  the  conversion  process.  This  minimized  any  extensive  note- 
taking on  the  part  of  the  Field  Artillery  School  lesson  developers  assigned 
to  the  workshop.  It  further  changed  and  simplified  the  nature  of  the  tasks 
involved  from  "learning  or  memorizing  a procedure"  to  "learning  to  follow 
the  steps  in  the  procedural  guide  or  job  aid  to  carry  out  the  procedure." 

This  was  further  simplified  to  show  the  stimulus  (cue  or  display)  for  each 
procedural  step,  the  action  taken,  and  the  feedback  which  results  from  the 
action.  This  is  illustrated  in  Appendix  B,  Figure  B-6,  which  shows  the  pro- 
cedures guide  to  run  (execute)  a CAI  lesson  under  the  course  developer  (author) 
control . 
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SME  Review  Procedure 

Block-3  Progress  Check 
Preparing  for  Student  Tryouts 

Determining  Revisions  Required 

Block-3  Progress  Check 


Section  4.  CONDUCT  OF  THE  TECMEDIA  CAI  WORKSHOP  AND  CONVERSION  COURSE 


A.  PRELIMINARY  ACTIVITIES 

Prior  to  the  start  of  the  workshop  and  conversion  course  (1  February  1977), 
the  requirements  were  defined  as  follows: 

1.  Dedicated  classroom  and  working  facilities  at  Fort  Sill. 

2.  ADDS  Consul  880  CRT/keyboard  and  slave  printer  (and  paper)  with  remote 
phone  tie  to  UNIVAC  1108  computer  at  Edgewood  Arsenal,  Maryland. 

3.  Assignment  of  four  U.S.  Army  Field  Artillery  School  personnel  for  the 
9-week  per iod . 

4.  PLANIT  CAI  system  on-line  on  the  UNIVAC  1108  computer. 

5.  Previously  prepared  CAI  materials  on-line  on  the  UNIVAC  1108,  i.e.. 
Lessons  INTRO  and  PLANIT,  and  the  CAI  LAW  lessons  with  lesson  listings. 

6.  Two  Beseler  Cue-See  devices. 

7.  TEC  filmstrips  and  audio  cartridges  available  for  the  materials  to  be 
converted,  including  Lesson  Administrative  Instructions,  Lesson  Pretest, 
and  Scoring  Key. 

8.  TEC  Final  Narration  scripts  and  other  production  backup  materials 
available  at  USAFAS. 

9.  A graphic  arts  and  reproduction  capability  for  the  several  off-line 
exhibits. 

The  TEC  materials  were  available  prior  to  the  start  date.  Copies  of  these 
materials  (scripts,  tests,  etc.)  were  reproduced  for  use  during  the  workshop 
and  conversion  course.  Materials  and  guidelines  on  course  development  for 
subject  matter  expert  review,  student  runs,  and  the  job  aids  and  procedures 
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guides  for  on-computer  use  were  finalized  and  reproduced  in  the  quantities 
required  for  the  9-week  workshop  and  conversion  course. 

Preparations  were  completed  by  28  January  1977. 

The  workshop  area.  Figure  4-1,  was  set  up  and  on-line  checkout  of  PLANIT, 

Text  Editor,  and  the  computer  system  were  made  on  31  January  1977.  Prepara- 
tions for  the  workshop  were  completed  on  schedule  and  ready  for  the  start  of 
the  workshop  on  the  next  day. 

B.  CONDUCT  OF  THE  TECMEDIA  CAI  WORKSHOP 
1.  Preparation  of  CAI  Lessons 

The  TECMEDIA  CAI  Workshop  was  conducted  in  Room  23,  Snow  Hall,  USAFAS,  Fort 
Sill,  Oklahoma.  The  four  USAFAS  course  participants  from  the  Directorate 
of  Course  Development  were  as  follows: 

Mr.  Robert  G.  Horn  (Team  1) 

Mr.  James  W.  Bowick  (Team  1) 

Mr.  Billy  J.  Hudson  (Team  2) 

Mr.  Douglas  M.  Converse  (Team  2) 

Mr.  Horn  was  designated  project  coordinator  by  Major  John  A.  Evans,  USAFAS 
Project  Officer.  Conducting  the  workshop  and  course  were  Dr.  William  G.  Hoyt, 
Mr.  Alfred  K.  Butler,  and  Mr.  Fred  D.  Bennik. 

The  workshop  was  performance  oriented  and  product  centered.  The  3-week 
schedule  shown  in  Appendix  E was  closely  but  not  rigidly  followed.  On  Day  1 
and  Day  2,  the  participants  received  hands-on  experience  with  TEC  and  CAI 
(LAW);  reviewed  the  TEC  materials,  Beseler  Cue-See  operation,  and  lesson 
scripts;  and  became  familiar  with  the  organization  and  execution  of  CAI  lessons. 
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By  Day  3,  both  teams  were  working  on  their  scripts  for  the  TEC  lesson  assigned, 
using  the  TEC-to-CAI  Conversion  Examples  Workbook  as  a procedures  guide.  Team 
1 (Figure  4-2)  had  "Determination  of  Direction,"  TEC  Lesson  No.  949-061-0001-F; 
and  Team  2 (Figure  4-3)  had  "Target  Location:  Polar  Plot  Method  and  Grid 
Coordinates  Method,"  TEC  Lesson  No.  949-061-0002-F.  The  course  developers 
developed  their  own  lessons  based  upon  the  workshop  discussions  and  the 
examples  given  in  the  workbook  and  CAI  listings.  This  included  the  answer 
processing,  branching,  decision  frames,  and  standards  of  performance  which 
they  considered  appropriate  for  the  lesson  and  tests  and  consistent  with  USAFAS 
standards  and  practices. 

On  Day  8,  the  teams  started  inputting  lesson  frames  into  the  computer  using 
the  Text  Editor  (Figure  4-4).  On  Day  11,  both  teams  had  prestored  and  built 
their  lesson  in  PLANIT,  with  Team  2 (Figure  4-5)  executing  and  editing  their 
lesson.  By  Day  14,  each  team  had  input  their  lesson  and  pre-  and  post  test, 
prestored  and  built  the  lesson  and  tests  in  PLANIT,  and  edited  and  executed 
them.  The  execution  runs  included  sequences  of  all  rights,  all  wrongs,  and 
combinations  of  rights  and  wrongs  to  exercise  the  various  decision  (D)  frames 
and  branching  which  had  been  included  by  the  lesson  developers. 

The  interaction  records  and  execution  (student)  records  for  these  author  runs 
provided  physical  evidence  that  a major  criteria  for  CAI  lesson  development 
had  been  met — that  the  lessons  execute  (run)  without  error  and  as  intended 
by  the  lesson  developer.  During  this  4-day  period,  including  Day  15,  the 
computer  was  down  each  day  for  periods  ranging  from  30  minutes  to  6 hours  for 
a total  of  10*5  hours.  Consequently,  the  subject  matter  expert  (SME)  review 
of  the  lessons  and  tests  took  place  on  Days  15  and  16. 
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2.  Subject  Matter  Expert  (SME)  Reviews 

The  purpose  of  the  SME  review  is  to  determine  whether  the  lessons  and  tests  are 
technically  accurate  and  complete,  and  are  consistent  with  current  or  upcoming 
Army  tactical  doctrine  and  employment. 

A second  purpose  is  to  further  verify  whether  the  objectives  of  each  CAI 
lesson  are  the  same  as  those  for  the  comparable  TEC  lessons. 

Three  SME  documents  were  pro  ced  by  SDC  for  conducting  the  reviews  during 
the  workshop  and  conversion  course.  These  are:  1.  Questionnaire  (SME); 

2.  Guidelines  for  Subject  Matter  Review;  and  3.  CAI  Lesson  Review.  These 
are  listed  in  Appendix  B,  Items  4,  5,  and  6. 


The  "Questionnaire  (SME)"  establishes  the  credibility  of  the  Subject  Matter 
Expert.  Basically,  it  asks  those  questions  relative  to  his  experience  and 
background  which  verify  that  he  is  proficient  in  the  subject  matter  area  being 
reviewed.  The  "Guidelines  for  Subject  Matter  Review"  give  direction  and 
structure  to  the  SME  for  his  review.  The  "CAI  Lesson  Review"  asks  those 
questions  necessary  to  verify  technical  content.  Army  doctrine,  lesson 
structure,  and  comparison  with  TEC. 


Upon  arrival,  the  SME  was  briefed  by  the  team  members,  including  a review  of 
the  objectives  of  the  TEC  lesson.  The  SME  filled  out  the  questionnaire,  was 
given  the  guidelines  to  use,  took  the  INTRO  lesson  to  familiarize  himself  with 
the  CAI  console  operation,  and  reviewed  the  lesson  and  tests  on-line  with  the 
team  members  (Figure  4-6).  Comments,  changes,  and  corrections  were  noted  on  the 
lesson  listings.  After  the  lesson,  the  SME  filled  out  the  "CAI  Lesson  Review" 
which  formed  the  basis  for  further  discussion  and  clarification.  The  review 
process  was  repeated  by  Team  2 on  Lesson  0002.  The  second  SME  (at  least  two 
for  each  lesson)  went  through  the  same  procedure. 


t. 


Figure  4-6. 


The  recorded  comments  of  the  SMKs  on  the  iesson  listings  and  the  CAI  Lesson 
Review  provided  physical  evident:  <;.t  t the  lessons  and  tests  were  technically 

accurate,  reflected  current  t o : i d e-trine,  and  covered  the  same  objectives 
as  the  comparable  TEC  lesson. 


Each  team  edited  and  executed  the  s i . son  and  tests  and  exhibits  based  upon 
the  SME  reviews.  Changes  to  the  lessons  (and  tests)  were  made  on-line,  and 
the  updated  lessons  were  immedi  < ! ready  anti  available  for  lesson  execution. 

The  2-day  SME  reviews  took  pi..  u m 15  and  16  with  editing  on  Day  16. 

The  lessons  were  ready  for  student  tryout  on  Day  17. 


3.  Student  .Tryouts  - Lessons  0001  and  0002 

The  purpose  of  the  student  tryout  is  to  determine  whether  learning  occurs  as 
a result  of  taking  the  lesson  as  measured  by  the  pre-  and  post  tests.  A 
second  purpose  is  to  "shake  down"  the  lessons  by  exposing  the  lesson  and  test 
situations  to  typical  students  to  determine  if  they  can  go  through  the  learning 
process  (take  the  lesson  and  tests)  easily  and  effectively.  If  not,  then 
identify  and  analyze  the  problem  areas  and  update  the  materials.  A third 
purpose  is  to  determine  student  attitudes  toward  the  CAI  materials. 

Two  documents  were  produced  by  SDC  for  use  during  student  tryouts.  These  are: 
(1)  "Questionnaire  (S),"  and  (2)  "CAI  Student  Attitude  Questionnaire."  These 
are  listed  in  Appendix  B,  Items  7 and  8. 

The  Questionnaire  (S)  is  used  primarily  to  verify  that  the  prerequisites  for 
taking  the  Observed  Fire  lessons  had  been  met,  i.e.,  the  student  had  already 
had  land  navigation  and  could  read  a military  map.  The  CAI  Student  Attitude 
Questionnaire  taps  the  student's  like  and  dislikes,  problem  areas  in  taking 
the  CAI  lessons,  and  level  of  confidence  in  being  able  to  carry  out  the  tasks 
learned . 

Student  runs  started  on  Day  17.  Upon  arrival  the  student  was  briefed  by  the 
team  members  and  filled  out  the  Questionnaire  (S).  The  student  took  the  INTRO 
lesson  to  familiarize  himself  with  the  CAI  console  operation.  He  then  took 
the  pretest  for  the  first  lesson.  If  he  failed  the  pretest,  he  took  the 
lesson  (Figure  4-7)  and  the  post  test.  If  he  passed,  he  took  the  pretest  for 
the  next  lesson.  Team  members  monitored  the  test  and  lesson  runs  by  means 
of  the  printer  which  provided  a complete  interaction  record  of  student  progress 
through  the  lesson.  When  the  post  test  was  completed,  the  student  filled  out 
the  CAI  Student  Attitude  Questionnaire.  PLANIT  student  records  of  the  pre- 
and  post  test  (Figure  4-8)  and  lesson  execution  (Figure  4-9)  were  obtained 
on-line  by  the  team  members. 
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Figure  4-7.  Student  Tryout 
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Figure  4-8.  PLAN  I.  T Student  Record-Tests 
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Figure  4-9.  PLANIT  Student  Record-Lessor  Execution 
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The  above  data  furnished  the  basis  for  team  members  to  discuss  the  lesson  run 
with  the  student.  Among  areas  covered  were  lesson  content  and  understanding, 
student  answers  to  questions,  off-line  exhibits,  lesson  execution,  and 
attitudes  of  student. 

The  student  runs  were  continued  until  the  desired  number  of  students  (four) 
had  been  run  for  each  lesson,  i.e.,  failed  the  pretest,  taken  the  lesson,  and 
passed  the  post  test.  Students  who  passed  the  pretest  were  not  included  in 
this  number. 

Changes  to  the  lessons  and  tests  were  effected  immediately  by  on-line  editing. 
These  changes  resulted  from  team  observation  of  the  student's  performance  on- 
line, discussions  with  the  student  on  problem  areas,  and  analysis  of  PLANIT 
student  records  as  to  constant  or  random  errors  made  by  the  students. 

The  lesson  and  test  listings,  interaction  records,  and  PLANIT  student  records 
were  used  to  identify  and  record  the  changes  to  be  made.  After  they  had 
completed  their  update,  the  team  members  obtained  new  listings  on-line.  Full 
advantage  was  taken  of  the  computerized  capability  to  automate  the  Instructional 
System  Development  (ISD)  process.  Student  runs  which  started  on  Day  17  were 
completed  on  Day  29. 

The  student  runs  for  the  workshop  overlapped  three  weeks  and  a day  into  the 
6-week  conversion  period.  The  student  runs  and  computer  input  of  new  lessons 
were  highly  dependent  on  the  one  CRT  console  (ADDS  terminal)  available. 

Priority  in  console  use  was  given  by  the  course  participants  to  student  runs. 

The  two  activities,  students'  runs  and  conversion  of  new  lessons,  continued  in 
parallel  until  the  student  runs  were  completed  on  Day  29  (1 A March).  Activities 
connected  with  student  runs,  e.g.,  analysis  of  student  records,  continued  into 
the  conversion  period. 
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4.  Problem  Areas  in  the  Workshop 

a.  Student  Selection.  A particular  problem  which  surfaced  at  this  time 
was  student  selection.  Eleven  students  were  processed  in  order  to  obtain 
data  on  the  desired  four  "usable"  students.  This  is  more  than  normally 
expected  if  student  selection  criteria  are  specifically  defined  and  the 
selection  process  adequately  carried  out  by  the  providing  organization.  One 
of  the  prerequisites  was  that  the  student  will  have  received  training  in  map 
reading  which  was  conducted  during  Basic  Combat  Training  (BCT) . It  turns  out 
that  map  reading  is  no  longer  covered  in  BCT  but  is  included  in  most  of  the 
Advanced  Individual  Training  (AIT)  courses.  Two  of  the  students  did  not  meet 
the  prerequisites  but  took  the  lessons  and  did  not  pass.  Other  factors  also 
were  evident.  Three  other  students  were  able  to  pass  the  pretest  for  the 
second  lesson.  Another  student  could  not  return  to  take  the  second  lesson. 
Students  were  also  not  available  every  day.  This  problem,  like  others  which 
came  up  in  the  9 weeks,  provided  the  course  participants  with  practical 
experience  and  insight  into  the  Instructional  System  Development  process.  As 
the  problems  came  up  they  were  discussed,  the  alternatives  based  upon  past 
experience  were  presented  along  with  the  probable  effects.  The  course  parti- 
cipants then  made  their  own  decisions.  This  was  an  integral  part  of  the  over- 
all learning  process  and  structure  of  the  course. 

b.  Computer  Down  Time.  Computer  down  time  (10*5  hours  over  4 days)  was 
not  considered  excessive  but  was  inconvenient.  It  occurred  primarily  during 
the  SME  and  student  on-line  executions  and  extended  the  schedule  and  time 
required. 

c.  One  CRT  Console.  The  one  CRT  console  (ADDS  terminal)  was  somewhat  of 
a bottleneck  as  the  course  participants  queued  to  get  on-line  to  input  their 
lessons  and  to  run  students.  This  also  limited  the  number  of  SMEs  and  students 
to  one  at  any  one  time.  This  was  alleviated  by  both  morning  and  afternoon 
on-line  sessions. 
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d.  Reference  Materials.  A minor  problem  occurred  in  regard  to  the 
references  cited.  This  is  a general  problem  in  the  ISD  process  and  not  specific 
to  TEC.  The  references  are  too  broad.  The  entire  FM  or  TC  is  cited  rather 
than  the  specific  pages  and  paragraphs  applicable  to  the  particular  task  or 
topic.  The  course  developer  (and  the  TEC  user)  has  to  research  the  document 
to  locate  the  material.  If  this  were  specified  to  begin  with,  time  would  be 
saved  and  there  would  be  no  errors  in  determining  and  locating  what  was 
applicable. 


The  reason  sometimes  given  for  not  being  specific  is  that  changes  are  made  to 
the  documents  which  make  the  reference  obsolete.  This,  however,  would  appear 
to  be  a valid  reason  for  being  specific. 

C.  CONDUCT  OF  THE  SIX  WEEK  TEC-TO-CAI  CONVERSION 
1.  Preparation  of  CAI  Materials 

In  the  workshop,  the  four  course  participants  worked  in  two  man  teams  on 
lessons  0001  and  0002.  They  continued  to  work  in  two  man  teams  conducting 
student  runs  on  lessons  0001  and  0002  until  these  were  completed  on  Day  29 
(14  March  1977). 


In  the  conversion  period.  Day  15  through  43,  each  course  participant  worked 
alone  and  was  solely  responsible  for  converting  his  own  lesson.  Assignments 
were  made  and  work  began  on  Day  15  (25  February)  as  follows: 


Individual 


TEC  Lesson 


Mr.  Horn 


Mr.  Bowick 
Mr.  Hudson 


Mr.  Converse 


Locate  a Target  by  Shift  from  a 
Known  Point,  //949-061-0003-F 

The  Call  for  Fire,  0949-061-0004-F 

The  Adjustment  of  Field  Artillery 
Fire  by  the  Bracketing  and  Creeping 
Methods,  Part  I,  //949-061-0005-F 

The  Adjustment  of  Field  Artillery 
Fire  by  the  Bracketing  and  Creeping 
Methods,  Part  II,  0949-061-0006-F 
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On  Day  20  (1  March),  five  work  days  after  start  of  the  conversion  period,  the 
first  lesson  frames  for  lesson  0006  were  input  using  the  Text  Editor.  By 
Day  33  (18  March)  all  four  lessons  and  eight  pre-  and  post  tests  for  lessons 
0003  to  0006  had  been  developed  and  input  into  the  computer.  By  Day  35,  20 
days  after  start,  all  lessons  had  been  prestored,  built,  edited,  and  executed 
in  PLANIT.  During  this  same  20  day  period,  11  of  the  days  had  included  SME 
and  student  runs  on  Lessons  0001  and  0002.  The  procedures,  procedures  guides, 
and  job  aids  used  during  the  conversion  period  were  the  same  as  those  used 
and  learned  in  the  workshop. 

2.  Subject  Matter  Expert  (SME)  Reviews 

The  SME  reviews  of  the  four  lessons  and  eight  pre-  and  post  tests  (12  PLANIT 
lessons)  took  place  on-line  on  Days  36  through  38  (23-25  March). 

Each  course  participant  conducted  the  SME  review  of  his  own  lesson  and  tests 
using  the  procedures,  questionnaires,  and  lesson  listings  as  described  earlier 
for  Lessons  0001  and  0002.  A total  of  three  SMEs  were  used  for  Lessons  0003 
to  0006  with  each  lesson  being  reviewed  independently  by  two  SMEs.  Lessons 
and  tests  were  edited,  based  upon  the  SME  reviews,  in  preparation  for  the 
student  tryouts  which  started  the  next  day.  Day  39. 


3.  Student  Tryouts  - Lessons  0003  to  0006 

Student  tryouts  of  the  four  lessons  and  eight  pre-  and  post  tests  began  on 
Day  39  (28  March) . 

Each  course  participant  conducted  the  student  tryouts  for  his  own  lesson  and 
tests  using  the  procedures,  questionnaires,  and  lesson  listings  as  described 
earlier  for  the  workshop  Lessons  0001  and  0002.  Changes  to  the  lessons  and 
tests  were  effected  immediately  by  on-line  editing.  These  changes  resulted 
from  observation  of  the  student's  performance  on-line,  discussions  with  the 
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student  on  problem  areas,  and  analysis  of  PLANIT  student  records  as  to 
constant  or  random  errors  made  by  the  students. 

The  lesson  and  test  listings,  interaction  records,  and  PLANIT  student  records 
were  used  to  identify  and  record  the  changes  to  be  made.  After  the  up-date 
was  completed,  new  listings  were  obtained  on-line.  Full  advantage  was  taken 
of  the  computerized  capability  to  automate  the  Instructional  System  Development 
(ISD)  process.  Student  runs  started  on  Day  39  (28  March  1977).  The  conversion 
period  officially  closed  1 April  with  five  students  having  been  run.  The 
course  participants  continued  to  run  students  after  1 April  to  obtain  the 
desired  number  (four)  for  each  lesson,  i.e.,  failed  the  pretest,  taken  the 
lesson,  and  passed  the  post  test.  Students  who  passed  the  pretest  were  not 
included  in  this  number. 
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Section  5.  RESULTS 

The  objectives  of  this  project  were  to: 

A.  Provide  a formative  evaluation  (preliminary  validation)  of  the  suitability 
of  CAI  as  one  means  for  providing  TEC  instruction. 

B.  Provide  documentation  of  CAI  to  TEC  conversion  costs  and  procedures  for 
conducting  a more  formalized  training  effectiveness  evaluation. 

C.  Determine  the  training  time  required  to  provide  Field  Artillery  School 
lesson  developers  with  the  capability  to  produce  CAI  versions  of  TEC. 

D.  Determine  the  amount  of  time  involved  and  types  of  difficulty  encountered 
in  having  Army  lesson  writers  convert  audio  visual  TEC  material  to 
appropriate  CAI  format. 

E.  Provide  three  alternative  versions  of  six  selected  TEC  lessons  (i.e.,  CAI 
stand-alone,  CAI/Audio-visual  combination.  Text  Presentation). 

F.  Provide  USAFAS  with  a course  developer  cadre  trained  in  the  practical 
steps  of  converting  self-paced  TEC  audio  visual  lessons,  and  in  use  of 
the  computer  system  to  input,  check,  edit,  and  update  CAI  materials. 

Convert  six  TEC  audio-visual  lessons  into  appropriate  CAI  format  and 
incorporate  all  identified  changes  in  doctrine,  employment,  etc.,  pertinent 
to  the  selected  TEC  lessons. 

The  results  obtained  in  regard  to  the  above  objectives  are  contained  in  the 

paragraphs  which  follow. 

A.  PRELIMINARY  EVALUATION  OF  THE  SUITABILITY  OF  CAI  AS  ONE  MEANS  FOR 
PROVIDING  TEC  INSTRUCTION 

The  preliminary  evaluation  cover  d three  basic  areas:  (1)  Do  the  lessons 

execute  (run),  (2)  Is  the  content  technically  accurate,  and  (3)  Do  students 

learn  as  evidenced  by  their  test  scores.  The  results  were  positive  as 
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described  in  paragraph  1.  which  follows.  Also  of  consideration  was:  Are  the 
lessons  acceptable,  i.e.,  what  is  the  attitude  of  those  concerned,  SMEs,  and 
students . Again,  the  results  were  positive  as  detailed  in  paragraphs  2.  and 
3.  which  follow. 

1.  Test  Scores 

Each  of  the  six  CAI  lessons  on  Observed  Fire  were  reviewed  by  Subject  Matter 
Experts  (SMEs)  for  technical  content  and  for  verification  of  current  tactical 
doctrine. 

Each  of  the  lessons  were  also  tried  out  (run)  on  a small  sample  of  students. 
Student  and  SME  runs  were  monitored  by  the  applicable  course  developer 
(teams  on  Lessons  0001  and  0002,  individuals  on  Lessons  0003  to  0006). 

Test  scores  for  the  first  18  students  (SI  through  S18)  on  which  the  lessons 
were  tried  out  are  shown  in  Table  5-1.  The  school  is  in  process  of  completing 
the  preliminary  evaluation  and  finalizing  the  lessons.  This  will  be  followed 
by  a full-scale  validation  of  30  students  on  all  six  lessons  (versus  the  six 
to  eight  for  each  lesson  in  Table  5-1). 

The  test  scores  show  that  learning  takes  place  as  a result  of  the  CAI  lessons. 

2.  Attitude  of  Students 

When  the  student  finished  his  lesson  and  tests,  he  completed  the  "CAI  Student 
Questionnaire"  (Appendix  D,  Item  8). 

The  results  show  that  students  like  this  method  of  instruction,  found  the 
lessons  easy  to  understand,  and  thought  the  method  of  instruction  was  effective. 
Most  of  them  did  not  express  any  problems  or  difficulties  in  interacting  with 
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♦Passed  pretest  so  did  not  take  lesson  or  post  test. 

♦♦Did  not  have  glasses. 

♦♦♦Prerequisites  not  met. 

DPasslng  score  Includes  passing  of  all  critical  Items  In  the  test 
2)Pretest  Increased  from  6 to  8 Items. 


the  computer.  The  comments  of  four  who  said  they  did  have  problems  are  as 
follows:  f? 


• At  first  I was  disoriented  and  didn't  take  enough  time. 

• Doing  before  thinking. 

• I was  misreading  problem  only  other  than  that  it  was  okay. 

Some  words  mean  something  different. 

• The  only  problems  was  pushing  the  key  button  with  wrong  answer 
and  not  being  able  to  stop  it. 

These  comments  do  indicate  that  the  lessons  required  them  to  take  time  and  to 
think  and  learn  about  what  they  were  doing. 


Other  comments  are  as  follows: 

• There  seems  to  be  a one-to-one  relationship  with  the  computer. 

You  go  back  over  the  question.  The  computer  will  tell  you  if  you 
were  incorrect.  You  don't  have  to  hold  up  a class  or  the  class 
doesn't  hold  you  up. 

• Instructors  can  be  boring  and  hard  to  listen  to.  You  can  do  this 
at  your  own  pace. 

• The  student  can  receive  the  lessons  almost  like  from  an  actual 
instructor  anytime  he  wants  to  or  has  time  to. 

• Because  it  holds  my  attention  well. 

• You  can  move  at  your  own  pace. 

• It  is  much  more  interesting  than  other  types. 

• This  type  of  training  allows  a man  to  work  at  his  own  speed  and 
holds  your  attention  better. 
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• If  I understood  something  I could  go  on;  if  not,  I could  go  over 
it  until  ready  to  go  on. 

• I believed  I learned  a lot  today  of  the  course. 

• I thought  they  were  very  informative  and  can  save  a great  deal 

of  time  in  learning.  More  work  with  a computer  of  this  type  will 
aid  a student  in  proficiency  and  comprehension. 

• With  the  right  programming  there  seems  to  be  no  limit  to  the 
different  subjects  you  could  learn. 

• It  keeps  you  interested  and  it  allows  you  to  advance  at  your 
own  pace. 

• It  explains  it  more  easier. 

• You  can  spend  time  to  think  and  figure  out  problems  without 
rushing  yourself. 

• It's  alright. 

• I enjoyed  taking  the  lesson. 

• I think  it  might  be  faster  than  the  classroom.  It  can  be  seen 
easier,  you  can  move  at  your  own  speed. 

• If  you  get  the  wrong  answer  you  can  go  over  it  right  away. 

• Because  it  is  very  easy  to  understand  and  the  facts  brought  forth 
were  straight  out,  very  fine  learning  process. 

e Using  the  computer  method  I think  holds  your  attention  to  the 
subject  being  presented,  and  knowing  your  wrong  it  gives  you  a 
chance  to  go  back  and  study  again. 

• I learned  a new  area  of  my  MOS. 

• I like  this  type  of  lesson  very  much  and  would  like  to  see  (it) 
put  into  use. 
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Even  though  they  liked  CAI  and  had  no  problems,  two  of  the  students  expressed 


preferences  in  regard  to  a specific  lesson  as  follows: 

• It  is  really  great  but  1 prefer  an  instructor. 

• I feel  that  it  would  be  an  aid  to  instruction  but  don't  feel 
it  could  replace  the  instructor. 
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In  summary,  the  response  to  the  questionnaires  showed  a positive  and  enthusiastic 
acceptance  of  this  method  of  instruction. 

3.  Attitude  of  SMEs 

The  SMEs  were  favorably  impressed  both  with  the  lessons  and  CAI  as  a media. 

The  SME  reviews  for  the  four  lessons  took  parts  of  three  days,  about  10  to  12 
hours.  The  review  was  done  "on-line"  with  comments  noted  on  the  course  mate- 
rials. After  the  review,  each  SME  filled  out  the  "CAI  Lesson  Review." 

(Appendix  D,  Item  6.)  Typical  comments  on  the  review  were:  

• It  was  thorough  and  easy  to  understand. 

• Standard  "FO"  procedures  were  followed  very  well. 

• The  CAI  is  a good  system,  much  better  for  teaching  and  maintaining 
student  interest.  It  won’t  let  you  quit — you  can  quit  the  TEC 

anytime. 

• The  lesson  was  accurate  and  thorough,  yet  maintained  a level  of 
learning  ability  for  the  average  soldier. 

• I don't  really  see  a problem  with  the  low  level  reader.  The 
lesson  was  simple  and  easy  to  understand  without  insulting  a man's 
intelligence. 

• Well .organized  lesson.  The  content  was  good,  precise,  and  presented 
in  a simple  easy  to  learn  format. 
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• This  lessoh  accurately  and  completely  covered  the  same  material 

that  the  TEC  lesson  covered.  • 

• As  for  completeness  and  accuracy,  both  the  TEC  lesson  and  the  CAI 
lesson  were  excellent,  but  the  CAI  lesson  was  a helluva  lot  more 
interesting. 

• It  moves  along  well  and  is  clearly  presented.  The  student  is  kept 
busy  throughout  the  program. 

• It  keeps  the  student  well  occupied  and  thinking. 

• Is  definitely  not  boring. 

B.  TEC-TO-CAI  CONVERSION  COSTS 

Daily  logs  were  maintained  by  each  of  the  four  course  participants  during  the 
workshop  and  the  conversion.  A daily  monitor's  log  was  also  maintained.  Also 
retained  were  costs  of  various  computer  runs.  In  addition,  a monthly  summary 
of  the  UNIVAC  1108  operating  expenses  was  obtained  for  the  months  of  February 
and  March  1977.  The  costs  are  based  largely  upon  the  four  lessons  (0003  to 
0006)  converted  during  the  six-week  conversion  period.  The  average  (or  mean) 
is  used  because  the  lessons  varied  in  length  (frame  size),  use  of  off-line 
exhibits,  and  execution  times.  The  averages  used  are  considered  representative 
of  typical  TEC  to  CAI  conversion  costs  for  a given  lesson. 
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The  mean  costs  derived  for  one  TEC  lesson  converted  to  CAI  Including  pre- 
liminary validation  with  four  usable  students  is  as  follows: 


1.  Mean  Hours  Required  (Per  Lesson  - Based  on  Four  Lessons) 

Course  Developer  - 111  hours  total 

SME  Review  (2  @ l*s  hrs.)  - 3 hours 

Students  (7  required  to  obtain  usable  - 14  hours 

sample  of  4,  2 hours  each  student 
including  questionnaires,  pre-  and 
post  tests  and  lesson). 

2.  Computer  Costs  (Derived  - Per  Lesson) 


Student  (11  hours)  and  SME  (3  hours) 

runs  - 14  hours  @ $7.50  - $ 105 

Inputting  lessons  in  Text  Editor  - 

16  hours  @ $1  16 

Checkout  and  editing  in  PLANIT, 
lesson  listings,  prestore, 

build,  restore  - 16  hours  @ $9.50  - 152 

Other  computer  costs  - 108 

Total  "$  381 


The  other  computer  costs  of  $108  include  facilities  usage,  catalogued  storage 
usage,  permanitizlng  PLANIT  lessons,  and  other  computer  uses  accociated  with 
the  project. 

3.  Telephone  Costs  (Derived) 

Telephone  cost  for  the  local  Baltimore  line  at  Fort  Sill,  Oklahoma,  is  a flat 
rate  of  $963  a month.  The  cost  of  the  line  is  based  upon  the  distance  from 
the  local  exchange.  For  the  two  months  of  the  project,  telephone  costs  were 
$1926.  Assuming  $1800  of  this  amount  was  due  to  the  six  lessons  (the  remaining 
$126  for  training)  this  would  prorate  the  telephone  costs  for  each  lesson  to 
approximately  $300. 


The  telephone  line  is  available  24  hours  a day,  seven  days  a week,  at  the 
same  cost  of  $963  a month. 
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The  computer  costs  and  telephone  costs  ($681)  shown  in  2.  and  3.  above  are 
based  upon  the  use  of  the  UNIVAC  1108  at  Edgewood  Arsenal,  Maryland.  Should 
Army  tactical  computers  be  used  for  this  purpose,  these  costs  would  not  be 
incurred.  There  would,  however,  be  time  associated  with  the  already  in  place 
Army  computer  operator  being  used  to  turn  the  system  on  (and  then  off)  when  it 
is  used  for  CAI . 

The  above  costs  do  not  include  secretarial  costs  as  there  are  few  associated 
with  the  development  of  CAI  courses.  The  course  developer  inputs  the  lesson 
into  the  computer  from  his  handwritten  script,  and  from  then  on  lesson  listings, 
scoring,  student  records,  etc.,  are  printed  by  the  computer.  When  off-line 
course  exhibits  are  included  (Lessons  0003  and  0004  had  exhibits.  Lessons  0005 
and  0006  did  not),  then  there  are  costs  associated  with  these.  The  lesson 
developers  produced  their  own  exhibits  for  use  during  the  lesson  development 
and  preliminary  evaluation. 

Calendar  time  (lead  time)  to  produce  course-ware  should  also  be  considered, 
as  well  as  the  time  required  to  update  lesson  materials.  The  calendar  time  to 
produce  the  four  lessons  is  approximately  30  working  days.  The  time  required 
to  update  (edit)  a lesson  when  a change  is  indicated,  e.g.,  a change  in 
tactical  doctrine,  is  a few  minutes  to  an  hour  or  so,  depending  on  the  number 
and  variety  of  changes  (editing)  to  be  made.  The  cost  savings  in  terms  of 
lead  time  in  meeting  training  needs  is  a factor  to  be  considered.  An  early 
lead  time  also  increases  the  useful  life  of  the  course  materials. 

There  are  costs  also  associated  with  the  management  of  course  development.  If 
lessons  are  produced  more  quickly,  with  less  preparation  of  paper  work,  then 
management  and  peripheral  costs  should  be  less. 

The  TEC  to  CAI  conversion  started  with  the  tasks,  training  objectives,  and  TEC 
lessons  and  scripts  available.  A reasonable  estimate  for  course  development, 


starting  from  the  task  and  training  analysis  at  the  beginning  of  the  ISD 
process  would  be  a 50%  or  60%  increase  in  the  lesson  preparation  stage. 

In  the  case  of  CAI,  this  would  require  an  additional  20  to  30  hours  to  be 
added  to  the  111  hours  of  course  developer  time.  The  remainder  of  the 
ISD  process  would  be  the.  same. 

C.  WORKSHOP  TRAINING  TIME  REQUIRED 

The  TECMEDLA  CAI  Workshop  and  Conversion  Course  was  set  up  in  two  parts.  The 
first  was  the  workshop,  Days  1 through  14,  in  which  initial  training  took 
place.  As  an  integral  part  of  the  initial  training,  two  TEC  lessons  were 
converted,  one  by  each  team  of  two  participants.  The  workshop  day  was  originally 
set  at  five  hours  (0730  to  1330,  with  a 1130  to  1230  break  for  lunch).  The 
student  runs  for  the  workshop  overlapped  and  ran  intermittently  parallel  with 
the  conversion  course  until  Day  29.  Each  team  was  responsible  for  the  complete 
ISD  process,  starting  with  the  tasks  and  objectives  (TEC)  having  been  defined. 
This  included  scripting,  updating  tactical  doctrine,  computer  input,  execution, 

c 

editing,  SME  reviews,  student  runs,  student  records,  analysis,  and  finalizing 
of  course  materials. 

The  workshop  training  time,  including  the  development  and  preliminary  validation 
(student  runs  through  Day  29)  for  Lessons  0001  and  0002,  was  as  follows: 

Total  Hours  Mean  Time  per  Team  Member 

Team  1 (Lesson  0001)  232  116 

Team  2 (Lesson  0002)  258  129 

Total  490  122.5 
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Of  the  490  preceding  hours,  approximately  20%  was  on  consul  (on  computer) 


time  as  follows: 

Total  Hours 

Mean  Hours  per  Team  Member 

Team  1 (Lesson  0001) 

57 

29 

Team  2 (Lesson  0002) 

42 

21 

Total 

99 

25 

Of  the  490  hours,  approximately  75%  was  involved  directly  in  doing  (hands  on) 
the  conversion  tasks  as  follows: 


Total  Hours 

Mean  Hours  per  Team  Member 

Team  1 

(Lesson  0001) 

184 

92 

Team  2 

(Lesson  0002) 

212 

106 

Total 

396 

99 

D.  TEC-TO-CAI  CONVERSION  TIME  AND  TYPES  OF  DIFFICULTIES  DURING  CONVERSION  COURSE 
1.  TEC-to-CAI  Conversion  Time 

In  the  six-week  conversion  course,  the  initial  training  received  in  the  workshop 
was  reinforced.  Each  course  participant  was  responsible  for  and  carried  out 
the  complete  process  (i.e.,  scripting,  doctrine  update,  SME  and  student  runs, 
etc.)  on  his  own  lesson.  The  six-week  conversion  course  officially  closed  on 
1 April  based  upon  agreement  by  the  ARI  COTR,  course  instructors,  course 
participants,  and  USAFAS  project  officer  that  the  course  participants  were 
indeed  fully  capable  and  confident  to  carry  out  the  remaining  student  runs. 

As  of  1 April,  five  students  had  been  run  on  Lessons  0003  to  0006.  Student 
runs  were  continued  by  the  course  participants,  and  by  7 April  seven 
students  had  been  run. 

On  7 April,  four  students  had  gone  through  the  pretest,  lesson,  and  passed 
the  post  test  for  Lessons  0003,  0004,  and  0005.  Two  had  done  so  for  Lesson 
0006.  It  is  assumed  that  no  more  than  four  additional  students  will  be 
required  to  complete  the  preliminary  evaluation  for  Lesson  0006,  the  remaining 
lesson.  A maximum  of  two  hours  per  student  run,  eight  hours,  is  estimated 
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for  this  effort.  The  TEC  to  CAI  conversion  times  for  Lessons  0003  to  0006, 
actuals  through  1 April  and  estimated  for  7 April  and  completion,  are  as 
follows: 


Lesson  No. 


Total  Hours 
to  1 April 


Estimated  Total  Hours 
to  7 April 


Estimated  Total  Hours  to 
Completion 


0003 

84 

87 

87 

0004 

110 

114 

114 

0005 

129 

133 

133 

0006 

97 

101 

109 

Total 

420 

435 

443 

Mean 

105 

109 

111 

The  mean  time  to  effect  the  TEC  to  CAI  conversion  process  during  the  course 
for  a given  lesson  from  initial  TEC  review  through  SME  and  preliminary 
validation  is  111  hours. 

A breakdown  of  the  111  hours  within  the  following  categories  used  is: 


overlap) 

Hours 

Percent 

36 

32% 

32 

29% 

43 

39% 

111 

100% 

Lesson  Preparation 

Initial  computer  input  and 
lesson  execution 

SME  review,  student  tryouts, 
and  editing 

Total 


2.  Types  of  Difficulties  Encountered 

In  spite  of  the  number  and  complexity  of  systems  involved  (Text  Editor,  PLANIT, 
UNIVAC  Executive,  and  Instructional  System  Development  (ISD)),  the  task  oriented, 
learner  centered  step-by-step  approach  led  to  few  problems  associated  with  the 
TEC  to  CAI  conversion  process.  The  procedures  guides  and  job  aids  were  a 
necessary  and  integral  part  of  this  process.  These  were  annotated  by  the  course 
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participants  as  necessary  to  integrate  them  into  the  process.  The  process  was 
also  simplified  in  that  once  the  lesson  was  input  into  the  computer,  the  computer 
took  care  of  most  of  the  paperwork  which  is  an  integral  part  of  course  develop- 
ment, i.e.,  lesson  listings,  interaction  records,  student  records. 

The  problems  that  did  occur  fell  into  two  areas,  computer  down  time  and  student 
selection  for  the  student  runs.  Computer  down  time  was  approximately  21  hours 
over  the  nine  weeks.  Problems  with  this  early  version  of  PLANIT  ARI  (Version  1.1) 
which  resulted  in  having  to  restore  the  system  accounted  for  approximately  15  hours 
of  downtime.  These  times  are  to  some  extent  maintenance  requirements  for  this 
version  of  PLANIT  and  not  considered  excessive.  They  were  somewhat  inconvenient, 
occurring  during  SME  review  and  student  runs.  They  did,  however,  provide 
opportunities  for  the  course  participants  to  exercise  their  skills  to  initiate 
the  UNIVAC  system  and  restore  PLANIT;  both  of  which  they  carried  out  with 
confidence. 

The  second  problem  was  student  selection.  This  problem  is  not  peculiar  to  CAI 
but  is  one  of  the  problems  in  carrying  out  the  validation  (preliminary  in  this 
case)  of  courseware.  For  example,  a total  of  11  students  for  Lessons  0001 
and  0002  were  required  to  obtain  four  usable  students  with  pretest  scores  low 
enough  to  require  taking  the  lesson  and  who  met  the  prerequisites  for  the 
lessons . 

During  this  period,  the  student  selection  process  was  reexamined.  It  was 
determined,  for  example,  that  land  navigation  and  map  reading  were  no  longer 
given  in  Basic  Combat  Training  (BCT) . Discussions  with  school  personnel 
responsible  for  TEC  resulted  in  clarification  of  TEC  prerequisites  for  these 
TEC  lessons  to  include  BCT  and  AIT  (map  reading  is  covered  in  most  AIT  courses) . 

The  student  selection  process  was  modified  accordingly. 
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E.  THREE  ALTERNATIVE  VERSIONS 

Three  alternative  versions  of  the  six  selected  Observed  Fire  TEC  lessons  were 
produced  along  with  the  modular  pre-  and  post  test.  These  are  identified  in 
the  following  paragraphs. 

1.  CAI  Stand-alone 

The  six  CAI  stand-alone  lessons,  with  their  pre-  and  post  tests,  were  produced 
by  the  four  course  participants  during  the  workshop  and  conversion  course. 
These  lessons  are  currently  available  on  computer  as  follows: 


PLANIT  Lesson  Name 


Title 


D0D1 

DODTA 

DODTB 

TL2 

TLTA 

TLTB 

TL3 

TATL3 

TBTL3 

CFF 

CFFTA 

CFTTB 

BCA 

BCATA 

BCATB 

BCB 

BCBTA 

BCBTB 


Determination  of  Direction 

Pretest 

Post  test 

Target  Location:  Polar  Plot  Method 
and  Grid  Coordinates  Method 
Pretest 
Post  test 

Locate  a Target  by  Shift  from  a Known  Point 

Pretest 

Post  test 

The  Call  for  Fire 

Pretest 

Post  test 

Area  Fire  Mission  (Bracketing  and 
Creeping  Methods)  Part  I 
Pretest 
Posttest 

Area  Fire  Mission  (Bracketing  and  Creeping 
Methods)  Part  II 
Pretest 
Post  test 
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2.  CAI/Audio-Visual  Combination 

The  six  CAI  lessons  given  in  1 above  along  with  the  following  TEC  audio-visual 
lessons  provide  the  CAI/Audio-Visual  Combination. 


TEC  Lesson  No.  Title 


1. 

#949-061-0001-F 

Determination  of  Direction 

2. 

#949-061-0002-F 

Target  Location:  Polar  Plot  Method  and 
Grid  Coordinates  Method 

3. 

#94 9-061-000 3-F 

Locate  a Target  by  Shift  from  a Known 
Point 

4. 

//949-061-0004-F 

The  Call  for  Fire 

5. 

//949-061-0005-F 

The  Adjustment  of  Field  Artillery  Fire 
by  the  Bracketing  and  Creeping 
Methods,  Part  I 

6. 

#949-061-0006-F 

The  Adjustment  of  Field  Artillery  Fire 
by  the  Bracketing  and  Creeping 

Methods,  Part  II 


The  TEC  lessons  will  be  updated  by  the  U.S.  Army  Field  Artillery  School  to 
reflect  current  tactical  doctrine  and  usage.  This  update  is  part  of  their 
TEC  review  and  update  cycle. 


3.  Text  Presentation 

The  six  Text  Presentation  lessons  (produced  by  project  personnel)  are  as 
follows: 


Lesson  1.  Determination  of  Direction 

Lesson  2.  Target  Location  - Polar  Plot  Method  and  Grid  Coordinates 
Method 


Lesson  3.  Locate  Target  by  Shift  from  a Known  Point 
Lesson  4.  Call  for  Fire 


Lesson  5.  Area  Fire  Missions:  Adjustment  of  Field  Artillery  Fire 
by  the  Bracketing  and  Creeping  Methods,  Part  I 

Lesson  6.  Area  Fire  Missions:  Adjustment  of  Fi*ld  Artillery  Fire 
by  the  Bracketing  and  Creeping  Methods,  Part  II 
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4.  Observed  Fire  Modular  Pre-  and  Post  Tests 

Project  personnel  also  developed  modular  pre—  and  post  tests  for  on- computer 
use  with  the  three  alternative  versions.  These  tests  are  currently  available 
on  computer,  as  follows: 

PLAN1T  Lesson  Name  Title 

OFTA  Observed  Fire  Pretest 

OFTB  Observed  Fire  Post  test 

F.  TRAINED  D SAFAS  CADRE 

Hie  four  TECMEDIA  CAI  Workshop  and  Conversion  Course  graduates  are  fully 
trained  and  provide  the  U.S.  Army  Field  Artillery  School  with  a trained  cadre 
of  personnel.  They  have  demonstrated  and  continue  to  demonstrate  their 
capability  to  convert  TEC  audio  visual  lessons  to  CAI.  The  six  TEC  lessons 
on  Observed  Fire  that  have  been  converted  to  CAT  attest  to  this  capability. 

The  six  CAI  lessons  are  current,  incorporating  the  latest  tactical  doctrine. 

The  four  course  graduates  have  demonstrated  and  continue  to  demonstrate  their 
ability  to  use  the  computer  system  to  input,  check,  edit,  and  update  CAI 
materials.  During  the  latter  stages  of  the  conversion  course,  they  were 
operating  and  interacting  with  the  computer  system  on  their  own.  Since 
1 April,  the  end  date  of  the  course,  they  have  continued  to  operate  the  system; 
run  students;  and  input,  check,  edit  and  update  their  CAI  materials.  This 
includes  all  aspects  of  the  Instructional  System  Development  process  involved 
in  TEC  to  CAI  conversion  including  data  collection,  data  compilation,  and 
data  analysis. 
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Section  6.  TEXT  PRESENTATION 


■ 


A.  INTRODUCTION 


One  of  the  project  objectives  (Section  2,  C,  5)  was  to  have  three  alternative 
versions  of  each  of  the  six  selected  TEC  Observed  Fire  lessons  made  available 
for  use  by  the  U.S.  Army  Field  Artillery  School.  Three  versions  were  (1)  CAI 
stand-alone,  (2)  CAI/Audio-visual  combination  (TEC),  and  (3)  Text  Presen- 
tation. These  alternatives  provide  USAFAS  with  a number  of  different  media 
and  combinations  of  media  for  the  six  Observed  Fire  lessons  to  meet  individual 
needs  or  preferences  of  students. 


The  CAI  stand-alone  version  was  produced  by  the  USAFAS  course  participants 
during  the  TECMEDIA  CAI  Workshop  and  Conversion  Course.  These  CAI  lessons  in 
conjunction  with  the  current  (and/or  to  be  updated)  TEC  lessons  on  Observed 
Fire  provided  the  CAI/Audio  visual  combination.  The  third  alternative.  Text 
Presentation,  was  produced  by  SDC  project  personnel  after  the  workshop  and 
conversion  course  was  completed. 


Development  of  the  Text  Presentation  materials  is  covered  in  Section  6.B., 
which  follows.  Development  of  the  modular  pre-  and  post  tests,  also  part  of 
the  alternative  version  requirements,  is  covered  in  Section  6.C. 


B.  PRODUCTION  OF  TEXT  LESSONS 


Text  lessons  are  printed  material  developed  to  serve  as  an  alternative  form  of 
self-paced  individualized  instruction.  Text  lessons  contain  the  tasks 
(objectives),  referenced  Army  publications,  supplemental  material,  test  items, 
and  a method  of  evaluating  performance.  The  requirements  specified  in  the 
six  Observed  Fire  TEC  lessons  (//949-061-0001-F  through  0006-F)  were  incorporated 
into  six  text  lessons  on  a one-for-one  basis.  For  each  of  these  text  lessons 
the  appropriate  reference  material  (FM,  TC,  etc.)  was  assembled  and  reviewed. 
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applicable  pages  from  each  publication  to  meet  the  objectives  stated  in  the 
TEC  lessons  were  reproduced,  and  test  items  to  assess  student  mastery  were 
developed . 

Supplemental  material  from  the  TEC-to-CAI  converted  lessons  were  also  incorp- 
orated into  the  text  lessons  to  cover  objectives  stated  in  the  TEC  lessons  that 
were  either  not  covered  or  incompletely  presented  in  the  Army  reference  ma- 
terial. The  use  of  supplemental  material  thus  brought  the  instructional  content 
of  the  text  lessons  in  concert  with  the  instructional  level  achieved  in  the  TEC 
and  CAI  lessons.  The  material  in  the  text  lessons  was  organized  as  follows: 


Introduction 


References 


Procedures 


Purpose  and  organization  of  the  lesson 
and  the  objectives  the  lesson  covers. 

Army  reference  material  and  listing  of 
all  applicable  pages  that  were  reproduced 
and  included  in  the  lesson.  Army  source 
documentation  included: 

FM  20-6  Map  Reading 

FM  6-40  Field  Artillery  Cannon  Gunr.ery 
FM  6-40-5  Modern  Battlefield  Cannon  Gunnery 
TC  6-40-4  Fire  for  Effect 

Instruction  on  how  to  proceed  through  the 
lesson. 


Topics  (Objectives  and  Reproduces  all  applicable  pages  from  the 

References)  Army  publications  indicated  in  the  reference 

section.  The  reference  pages  from  all  source 
material  for  each  objective  are  gathered 
together.  This  is  followed  by  supplemental 
material  extracted  from  the  applicable  CAI 
lesson.  Extracted  CAI  material  is  reworded 
as  appropriate  from  an  on-line  interactive 
form  of  presentation  to  conform  to  a more 
conventional  non-conversational  printed 
text  format. 


Performance  Quiz 


Scoring  Key 


Self  Evaluation 


Presents  the  performance  test.  The  test 
items  are  performance  oriented  set  in  a 
situational  context.  Instructions  on  how 
to  answer  the  test  items  are  included  in 
the  stem.  Multiple-choice  questions,  when 
used,  usually  contain  four  alternative 
answer-choices . 

Provides  a scoring  key  for  self  scoring  the 
test,  indicating  the  number  of  possible 
points  for  each  item  or  sub  item. 

Specifies  the  criteria  for  successfully 
meeting  the  objectives  of  the  lesson. 
Instructions  are  provided  to  guide  the 
student  on  to  his  next  step — proceed  or 
enter  a review  cycle. 


The  objectiv  3,  number  of  performance  test  items,  possible  points  and  passing 
score  developed  for  each  text  lesson  are  summarized  in  Table  6-1. 


C.  OBSERVED  FIRE  MODULE  PRE-  AND  POST  TESTS 


The  Observed  Fire  Module  Pre-  and  Post  Tests  (OFTA  and  0TTB  - separate  PLANIT 
lessons)  were  developed  by  project  personnel  to  assess  student  performance 
as  part  of  Task  4,  Three  Alternative  Versions  (Section  2,  C,  5).  The  module 
pre-  and  post  tests  are  given  on-line  (on  computer)  and  are  automatically 
scored.  Decision  frames  have  been  included  which  determine  for  the  student 
which  of  the  six  Observed  Fire  Lessons,  if  any,  he  has  passed  or  needs  to 
take.  The  module  pre-  and  post  tests  were  developed  by  providing  alternate 
forms  of  the  pre-  and  post  test  items  (one  for  one)  that  had  been  generated 
in  the  workshop  and  conversion  course  to  assess  student  performance  for  the 
six  CAI  lessons.  These  lesson  assessment  test  items  were  based  in  turn  on 
test  items  developed  for  the  individual  TEC  lessons.  The  alternate  forms  of 
the  test  items  were  then  chained  together  to  form  separate  pre-  (OFTA)  and 
post  module  (OFTB)  tests. 


Items  selected  from  the  CAI  lesson  tests  were  treated  as  follows: 


• The  content  (values,  names,  etc.)  of  Items  was  changed  whenever 
possible  for  each  test. 

• Items  requiring  constructed  responses  that  did  not  lend  themselves 
to  alteration  were  inserted  at  different  points  within  each  test. 

• The  order  was  scrambled  for  alternatives  in  multiple-choice  items 
that  could  not  be  changed. 

The  number  of  items  in  OFTA  and  OFTB  and  the  PLANIT  CAI  lessons  from  which 
they  were  derived  are  shown  below.  The  number  of  items  are  the  sane  as  in 
the  CAI  lessons  prepared  by  the  course  participants,  e.g.  the  DOD  pretest, 
DOTA,  has  8 items. 


» — 

LESSON 

PRETEST  (OFTA) 

POST  TEST  (OFTB) 

D0D1 

8 

8 

TL2 

6 

6 

TL3 

9 

3 

CFF 

6 

6 

BCA 

6 

6 

BCB 

5 

5 

Total  Items 

40 

34 

Adjunct  materials  (off-line  exhibits)  were  developed  in  conjunction  with  the 
construction  of  each  module  test.  Test  items  that  required  an  accompanying 
exhibit  that  could  not  be  accomplished  on-line  were  flagged  as  needing  off- 
line materials.  Development  of  these  materials  was  accomplished  either  by 
securing  copies  of  pictures  and  diagrams  that  were  contained  in  TEC  source 
\ documentation  and  modifying  them  as  required,  or  by  preparing  original 

material.  Completed  materials  for  each  module  test  were  bound  separately 


as  handouts  for  student  use. 


The  strategy  of  each  module  test  Is  to  exploit  the  diagnostic  capabilities 
of  the  CAI  language  PLANIT  followed  by  outputting  prescriptive  information 
to  the  student  concerning  the  performance.  This  is  accomplished  as  follows: 
The  student  takes  the  module  test  (pre-  or  post)  and  proceeds  through  the 
test  item-by-item.  At  the  completion  of  the  test,  the  students'  performance 
is  evaluated  and  based  upon  pre-determined  decision  criteria,  prescriptive 
information  is  displayed  to  the  student.  Lessons  for  which  the  student  has 
met  criteria  are  indicated  as  "passed"  and  need  not  be  taken  (pretest)  or 
repeated  (post  test).  When  the  student  fails  to  meet  criteria  for  individual 
lessons  he  is  informed  which  lesson(s)  he  must  take  (pretest)  or  repeat  (post 
test).  Lessons  flagged  to  be  taken  are  available  to  the  student  in  three  dif- 
ferent media — TEC,  CAI  or  as  Text  materials,  or  combinations  of  these. 


Section  7.  CONCLUSIONS  AND  RECOMMENDATIONS 


The  following  conclusions  and  recommendations  are  based  on  the  results  of 
conducting  the  TECMEDIA  CAI  Workshop  and  Conversion  Course. 

A.  CONCLUSIONS 

1.  A preliminary  evaluation  of  the  six  Observed  Fire  CAI  lessons  was  conducted. 
Results  show  students  learn  and  like  the  CAI  lessons. 

2.  The  CAI  to  TEC  conversion  costs  are  summarized  as  follows.  The  costs 
include  subject  matter  expert  (SME)  review  and  preliminary  evaluation 
(small  group  tryout)  based  on  four  usable  students  being  obtained.  The 
costs  are  based  upon  an  average  lesson  of  approximately  l*t  hours, 
including  pretest  and  post  test. 

a.  Total  Man-Hours  (per  lesson) 

Course  Developer  - 111  hours 

SME  Review 

(2  @ l*s  hrs.  each)  - 3 hours 

Student  hours  for  pre- 
liminary evaluation 

(7  students  @ 2 hrs.  each)  - 14  hours 


b.  Computer  Costs  (per  lesson) 

Student  (11  hours)  and  SME 
(3  hours)  runs,  14  hours 
@ $7.50  - $ 105 

Inputting  lessons  in  Text 

Editor  - 16  hours  @ $1  - 16 

Checkout  and  Editing  in 
PLANIT,  lesson  listings, 
prestore,  build,  restore, 

16  hours  @ $9.50  - 152 

Other  computer  costs  - 108 

Total  $ 381 


The  other  computer  costs  of  $108  include  facilities  usage,  catalogue  storage 
usage,  permanitizing  PLANIT  lessons,  and  other  computer  uses  associated  with 
the  project. 
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c.  Telephone  Costs  (Local  Baltimore  line  at  Fort  Sill,  flat  rate) 

Telephone  costs  for  the  local  Baltimore  line  at  Fort  Sill  for  the  two 
months  of  the  project  were  $1926  (flat  rate  of  $963  per  month). 

Assuming  $1800  of  this  amount  was  due  to  the  six  lessons  (the  remaining 
$126  for  training) , this  would  prorate  the  telephone  costs  for  each 
lesson  to  approximately  $300. 

3.  The  training  time  required  to  provide  each  of  the  four  Field  Artillery 
School  lesson  developers  with  the  capability  to  produce  PLANIT  CAI  versions 
of  TEC  (or  new  CAI  materials)  is  123  hours  (total  490  hours).  As  part  of 
the  490  hours,  two  TEC  lessons  were  converted  to  CAI,  reviewed  by  subject 
matter  experts,  and  a preliminary  evaluation  conducted  using  Army  subjects. 
Full  use  was  made  of  job  aids  and  procedures  guides  developed  for  the 
conversion  course. 

4.  Using  CAI,  USAFAS  course  developers  can  produce  and  evaluate  lessons  in 
approximately  111  course  developer  hours  per  lesson.  Difficulties 
encountered  were  minor  and  in  two  areas  peripherally  related  to  the 
project;  computer  down  time,  and  selection  of  tryout  students.  Better 
selection  procedures  and  coordination  have  alleviated  the  latter. 

5.  As  a result  of  this  project,  the  USAFAS  has  the  six  "Observed  Fire" 
lessons  in  three  alternative  versions,  CAI  stand-alone,  audio  visual  (TEC), 
and  Text  Presentation  (printed  format),  and  combinations  of  these.  In 
addition,  module  pre-  and  post  tests  for  on-line  computer  administration 
have  been  developed  for  the  six  lessons. 

6.  The  U.S.  Army  Field  Artillery  School  (USAFAS),  as  a result  of  this  project, 
has  a cadre  of  trained  personnel  who  have  demonstrated  and  continue  to 
demonstrate  their  capability  to  convert  TEC  lessons  into  CAI  and  to  use 
the  UNIVAC  computer  system  and  PLANIT  CAI  system  to  develop,  execute, 
edit,  update,  and  validate  CAI  lesson  materials. 
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B.  RECOMMENDATIONS 


The  recommendations  which  follow  are  those  applicable  in  the  evaluation  of 
CAI  as  a delivery  system  for  TEC  Army-wide  (including  USAFAS)  and  those  which 
have  a specific  potential  for  USAFAS  applications.  Recommendations  1 through  3 
are  in  regard  to  the  CAI  application  Army-wide  and  4 through  7 are  more  specific 
to  USAFAS  applications. 

1.  The  TECMEDIA  Workshop  and  Conversion  Course  has  demonstrated  thac  USAFAS 
Course  Development  personnel  have  quickly  developed  a capability  to 
produce  CAI  course  materials.  In  producing  the  six  Observed  Fire  CAI 
lessons,  they  have  followed  the  ISD  model  from  the  point  where  the  tasks 
and  training  analysis  has  been  completed  and  scripts  produced.  The 
scripts  were  revised  and  new  material  added  as  a result  of  changes  in 
tactical  doctrine,  organization,  and  language.  As  part  of  this  process, 
they  have  learned  and  experienced  the  strengths  and  weaknesses  of  CAI 
as  compared  with  other  systems  and  have  concluded  that  CAI  will  become 
an  important  factor  for  Army  training.  They  are  also  capable,  as  a 
result  of  their  experience,  to  evaluate  how  and  where  CAI  should  be  used 
and  the  factors  to  consider  in  implementing  CAI  systems.  This  is  a 
capability  which  other  Army  schools  should  have. 

It  is  recommended  the  TECMEDIA  CAI  Workshop  and  Conversion  Course  be 
given  at  other  Army  schools  for  course  development,  evaluation,  and 
training  management  personnel  to  develop  the  same  capability,  produce 
CAI  courseware  in  their  subject  areas  of  responsibility,  and  provide 
experience  to  evaluate  the  usefulness  of  CAI  as  a training  media.  The 
result  would  be  a common  pool  of  lessons  prepared  by  course  development 
personnel  (i.e.,  the  responsible  branch  school)  which  would  be  readily 
available  to  other  branch  schools,  operational  units,  reserves,  and 
national  guard. 
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2.  A clearly  demonstrated  advantage  of  CAI  is  the  speed  and  ease  in  carrying 
out  the  Instructional  System  Development  (ISD)  process.  Much  of  this 
can  be  attributed  to  the  computerization  of  myriad  paper  work  usually 
involved.  In  CAI,  the  lesson  developer  inputs  his  lessons  into  the 
computer  from  his  handwritten  scripts.  From  then  on,  listings,  SME 
reviews,  student  records,  scoring,  etc.,  are  automated.  Update  of 
lesson  materials  based  upon  the  ISD  process  is  done  on  computer,  with 
the  lessons  immediately  ready  for  the  next  step.  The  manual  preparation 
of  course  materials  is  no  longer  required,  the  exception  being  off-line 
course  exhibits  when  these  are  needed  for  the  lesson. 

It  is  recommended  that  the  costs  and  times  associated  with  the  development 
of  CAI  materials  be  compared  with  the  costs  and  times  associated  with 
preparation  of  other  materials.  The  comparison  to  include  low  density 
and  high  density  expected  use  of  lessons  to  determine  cross-over  points 
in  regard  to  cost  effectiveness.  Low  development  costs  and  high  usage 
costs  are  cost  effective  when  the  student  use  (density)  is  low  (one  curve) 
as  compared  with  high  development  costs  and  low  usage  costs  when  the 
student  use  (density)  is  high  (second  curve).  At  some  point  the  two 
curves  can  be  expected  to  cross.  Computer  costs  include  storage  and 
distribution.  Lead  times  for  course  materials  and  paper  work  costs  for 
lesson  development  should  be  part  of  the  comparison. 

3.  The  Observed  Fire  series  of  lessons  provide  USAFAS  with  a three  track 
capability — CAI,  audio  visual  (when  TEC  is  updated),  and  printed  material 
(Text).  It  is  also  clear  that  students  vary  in  their  preferences  for 
the  different  type  of  media. 

It  is  recommended  that  the  three  tracks  be  made  available  for  student 
use  and  research  conducted  to  establish  the  relationships  between 
student  characteristics,  media,  and  learning  effectiveness  as  it  relates 
to  the  student/media  (aptitude  x treatment)  interactions. 


7-4 


'1 


/ 


4.  The  U.S.  Army  Field  Artillery  School  has  developed  the  series  of  six 
Observed  Fire  CAI  lessons.  Observed  Fire  is  a requirement  for  all  combat 
arms  personnel. 

It  is  recommended  that  the  USAFAS  complete  the  ISD  process  by  conducting 
the  large  group  tryouts  to  validate  the  Observed  Fire  lessons  and  that 
they  be  made  available  to  all  the  combat  arms.  Validation  could  be 
conducted  directly  through  the  computerized  configuration  presently  being 
used,  but  remoted  to  other  users,  e.g.,  at  Fort  Hood.  Further,  the 
lessons  could  be  adapted  to  the  parameters  required  by  the  TACFIRE  computer 
and  then  presented  on  tactical  data  systems  should  such  a policy  decision 
be  made  by  the  Army. 

5.  The  U.S.  Army  Field  Artillery  School  has  converted  the  six  Observed  Fire 
TEC  lessons  to  CAI  and  has  the  capability  to  convert  other  TEC  lessons  or 

to  develop  new  CAI  lesson  materials.  The  process  for  converting  current  TEC 
materials  or  developing  new  materials  is  the  same,  except  that  in  the  case 
of  new  materials,  the  front  end  of  the  ISD  process,  task  and  training 
analysis,  needs  to  be  conducted.  The  front  end  analysis  is  part  of  the 
ISD  process  and  is  independent  of  the  media  used. 

It  is  recommended  that  the  U.S.  Army  Field  Artillery  School  initially 
develop  additional  CAI  lessons  in  low  density  student  areas  that  have 
problems  in  terms  of  complexity  of  material,  potential  shortage  of 
instructors,  high  failure  rate  or  a high  need  for  remedial  training. 
Suggested  areas  are  FADAC  maintenance  and  operation.  Fire  Support  Team 
Operation,  Pershing  repair,  SR-56  operation,  and  survey. 


7-5 


6.  The  current  CAI  system  configuration  utilizes  a single  student  terminal 
coupled  to  an  earlier  version  (ARI  Version  1.1)  of  PLANIT.  This  con- 
figuration restricts  the  CAI  system  to  single  student  usage  at  one  time 
and  requires  additional  maintenance  steps  (and  time)  to  increase  the 
smoothness  of  operation  by  eliminating  spurious  PLANIT  errors.  This 
limitation  places  an  additional  burden  on  the  USAFAS  CAI  trained  cadre 
to  provide  an  error  free  system  during  course  development  and  subsequent 
student  system  use. 

It  is  recommended  that  a more  recent  ARI  version  of  PLANIT  (not  previously 
available)  be  coupled  with  the  existing  USAFAS  CAI  system  to  enable  the 
practicality  of  the  system  to  be  increased  by  permitting  an  increase  in 
the  number  of  student  terminals.  The  latter  CAI  system  expansion  would 
permit  a greater  number  of  students  to  be  run  while  decreasing  the 
calendar  time  period  required. 

7.  The  USAFAS  has  AO  hours  of  CAI  TACFIRE  courseware  covering  the  operation 
of  the  Fire  Direction  Center  Artillery  Control  Console  (ACC) . This 
courseware  was  produced  several  years  ago,  and  changes  in  tactical 
doctrine  and  employment  have  occurred.  The  course  materials  need  to  be 
updated . 

It  is  recommended  that  the  USAFAS  Course  Development  personnel  update  the 
existing  40  hours  of  TACFIRE  CAI  courseware. 
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The  scope  of  this  project  was  as  follows: 


1.  Assist  personnel  of  the  USAFAS  in  converting  TEC  materials  into  CAI  format. 
All  CAI  lessons  will  conform  to  the  stated  objectives  of  the  self-paced 
TEC  lessons. 

2.  Train  USAFAS  lesson  writers  in  the  practical  steps  of  converting  from 
TEC  audio-visual  lessons  to  CAI  lesson  materials  during  a three-week 
workshop.  Two  CAI  lessons  will  be  converted  during  this  period.  Four 
more  TEC  lessons  will  be  converted  during  the  subsequent  six-week 
conversion  period  following  the  workshop. 

3.  During  the  TEC-to-CAI  conversion  process  the  CAI  lessons  constructed 
will  incorporate  identified  changes  required  to  the  TEC  lessons,  i.e., 
changes  in  tactical  doctrine,  employment,  etc. 

4.  Make  available  three  alternative  versions  (i.e.,  CAI  stand-alone, 
CAI/Audio-Visual  Combination,  Text  Presentation)  of  each  of  the  selected 
TEC  lessons  for  use  by  the  USAFAS. 

5.  A preliminary  validation  of  the  CAI  materials,  employing  subject  matter 
experts  (lesson  evaluators)  and  naive  student  subjects,  will  be  conducted. 

6.  A data  collection  system  will  be  designed  which  will  permit  very  accurate 
records  to  be  kept  with  respect  to  the  length  of  time  spent,  as  well  as 
the  nature  and  amount  of  difficulty  encountered  when  converting  TEC 
materials  to  CAI  format. 

7.  A plan  will  be  produced  which  will  indicate  how  the  Field  Afillery  School's 
resources  could  best  be  utilized  to  develop  strategies  for  media  selection 
in  the  training  of  field  artillery  personnel. 
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The  lessons  used  In  this  project  were  selected  by  the  U.S.  Army  Field 
Artillery  School.  The  criteria  established  for  selection  were: 

a.  The  material  must  already  exist  in  TEC  format. 

b.  The  material  should  be  relevant  to  several  MOS's. 

c.  The  material  should  be  readily  amenable  to  performance  testing. 

d.  The  material  should  be  representative  of  different  types  of  skills. 

The  TEC  lessons  selected  were  the  series  of  six  on  Observed  Fire  as  follows 

a.  Determination  of  Direction,  TEC  Lesson  //949-061-0001-F. 

b.  Target  Location:  Polar  Plot  Method  and  Grid  Coordinates  Method, 

TEC  Lesson  0949-061-0002-F. 

c.  Locate  a Target  by  Shift  from  a Known  Point,  TEC  Lesson  //949-061-0003-F 

d.  The  Call  for  Fire,  TEC  Lesson  //949-061-0004-F. 

e.  The  Adjustment  of  Field  Artillery  Fire  by  the  Bracketing  and  Creeping 
Methods,  Part  1,  TEC  Lesson  //949-061-0005-F. 

f.  The  Adjustment  of  Field  Artillery  Fire  by  the  Bracketing  and  Creeping 
Methods,  Part  2,  TEC  Lesson  //949-061-0006-F. 
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APPENDIX  B 

PROCEDURE  GUIDES  FOR  ACCESSING  AND  OPERATING  THE 
UNIVAC  1108/EXEC  8,  TEXT  EDITOR  AND  PLANIT  SYSTEMS 
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A/O  T 

ft-ecedwG  ' F/ l Ate J , 


HOW  TO  SIGN  ON  TO  THE  UN I VAC  SYSTEM  I 


Cue  or  Display 


Your  Action 


CONNECT  CONSOLE  TO  COMPUTER 


CRT  screen  light* 
Printer  hums 
High  pitched  ton* 

CRT  carrier  light  ON 


Edgcwood  Arsenal,  etc. 

(appears  on  CRT  screen) 


Obtain  authorized  UNIVAC  & 
PLANIT  Identifiers 

Turn  CRT  power  switch  ON 
Press  CONV  Mode  key 

Press  print  line  button 
Dial  phone  number 
Place  phone  In  coupler 

Key  IARI0S  or  ARI06 
Press  NEW  LINE  key  (NL) 


- See  Trainer  or  Monitor 


- to  activate  CRT  and  keyboard 

- puts  console  In  Interactive 
■ode 

- to  activate  printer 

- to  get  coaputer  channel 

- to  permit  data  transfer 

- Identifies  your  console 
transmits  to  computer 

you  are  on-computer 


DATE:  DDMMYY  TIME:HH>ttSS 
e.g.,  130277  074330 

> appears  on  screen 


DECLARE  JOB  TO  EXECUTIVE  SYSTEM 


key  9RUN(space)Name, (account  number) 
e.g.  9RUN  HORN, (account  number) 

Press  NEW  LINE  key  (NL) 


key  eeCQUE  (NL) 


> will  no  longer  appear 


You  can  now  access  either  the  UNIVAC  Text  Editor 
or  the  PLANIT  CAI  System 


TO  CORRECT  YOUR  KEYING  ERRORS  BEFORE  ENTRY 


Incorrect  typing 


Cursor  j*ps  to  next 
line,  left  margin  of 
screen 


Hold  CONTROL  key  down  and  then 
press  X key  (before  pressing 
NEW  LINE  key) 

Key  new  replacement  line  (NL) 


to  override  keyed  line 
before  transmit  to  computer 

transmit  corrected  entry 


Figure  B-l.  Procedure  Guide  to  Sign  on  to  the  UNIVAC  System 


Cue  or  Display 


(several  Unas  then) 

11: 


(Prints  line  mjnber) 


(Prints  line  nusiber) 
0: 


(Prints  line  nueber) 


(Prints  total  nuaber 
of  lisas  in  file) 


BUILDING  THE  CARD  INPUT  FILE 


Your  Action 

«D,I  TECMEDIA*H0RN.CCF1  (NL) 


KEY  IN  DATA  A LINE  AT  A TIME 
Last  line  Is  $$$$$$ 


(ML) 


TOP  (NL) 
LNPRINT.'  (NL) 


EXIT  (NL) 


sent 


Call  Text  Editor  to 
create  a new  lesson 
card  file. 


This  builds  dats  file. 


Blank  line  stops 
Input  aode. 


Go  to  top  of  file. 

List  total  file  at  CRT 
and  on  llnepr Inter. 

Leave  Text  Editor. 


Figure  B-2.  Procedure  Guide  for  Use  of  Text 
Editor  - Input  Mode 
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UPDATING  A PERMANENT  FILE 
USING  TEXT  EDITOR 


Cue  or  Display 

Your  Action 

Consents 

0ED.U  TECMEDIA*H0RN . CFF1  (NL) 

call  up  the  text  editor 

display  several  lines, 
then 

0: 

AUTO  (NL) 

INITIALIZE  SAFETY  FEATURE 

0: 

GOTO  145  (NL) 

Position  editor  to  line  145 

display  text  for  line 

145,  then 

145: 

D (NL) 

deletes  the  line  positioned 

at 

145: 

P (NL) 

print  the  line  positioned 

at 

displays  line  contents 
145: 

R+  (NL) 

Replace  the  line  positioned 
at 

•f 

text  for  replaceaent  (NL) 

New  line  contents 

145: 

1+  (NL) 

Input  a new  line  of  text 
after  the  line  positioned 

at 

♦ 

text  for  new  line  (NL) 

Inserted  line 

145: 

IB+  (NL) 

Input  a new  line  of  text 
just  before  the  line 
positioned  at 

♦ 

text  for  new  line  (NL) 

new  Inserted  Before  line 

145: 

C /OLDTEXT/NEWTEXT/  (NL) 

Change  the  old  text  to 
new  text  at  the  line 
positioned  at  only 

Figure  B-3.  Procedure  Guide  for  Use  of  Text 
Editor  - Edit  Mode 
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A 


HOW  TO  USE  PLANIT  TO  CONSTRUCT  A 
LESSON  PROM  TEXT  EDITOR  LESSON  FILE 


Cue  or  Display 


Tour  Action 


Completion  of  a laaaon 
card  images  file. 


String  of  READY  massages 
ending  with:  LOGIN  OR  END 


ENTER  COWHAND 


ENTER  SYSTEM  COWHAND 


ACCESS  PLANIT  SUPPORT  MODE 


0ADD  PLANIT*PRUNS.HOT  (NL) 


Identifier  (NL) 


A SYSTEM  (NL) 


- to  load  PLANIT 

(PLANIT  loaded) 

- get  identifier  from 
monitor 


to  Support  Mode 
(Support  Mode  prompt) 


I READ  LESSON  CARDS  FILE] 


PRESTORE  Filename  (NL) 
e.g.  PRESTORE  OB14768  (NL) 


- use  Text  Editor 
cards  file  name, 
(card  Images  read) 


SSONFOR  PLANIT 


Error  Messages 
(if  any)  on  screen 
A printer,  then: 

S 


IDENTIFY  YOURSELF*** 


BUILD  Lessonnaae  (NL) 
e.g.  BUILD  0BFIRE1  (NL) 


SAVE 

e.g.  SAVE  OBFIREl 


identifier  (NL) 


to  check  cards  A 
build  lesson  for 
PLANIT. 

(lesson  constructed) 
to  name  and  Inventory 
the  lesson 


- your  authorised 
author  identifier 


PLANIT  lessons  list 
on  screen  A printer 


UNLOCK  (NL) 

A SYSTEM  (NL) 
LIST  $TOC  (NL) 


- to  be  able  to  access 
lesson 

- to  check  If  lesson 
is  built 

Check  lesson  name,  your 
ID,  A UNLOCKed  status 


Figure  B-4.  Procedure  Guide  to  Build  a 
Lesson  Using  PLANIT 


B-6 


Cue  or  Display 


Your  Action 


Comment 


© |To  Replace  Frame  Code  I 

ENTER  COMMAND 

* frame,  group,  line,  E (NL) 

e.g.  2,  3,  1,  E (NL) 


(Line  to  be  replaced 
appear*  on  ecreen.) 

Newline  (NL) 


- to  replace  a 
line  (replace 
Line  1,  in 
Croup  3,  Frame  2) 


Enter  new  line 


frame,  group,  E (NL) 
e.g.  2,  3,  E (NL) 


(Group  proapt 

message  appears) 


a 

* 


ENTER  COMMAND 


new  group  line  1 
line  2 


etc. 

& (NL) 


- to  replace  a 
group  (replace 
Group  3 in 
Frame  2) 


enter  lines  of 
new  group 

when  finished 


Q/M/D/P 

* 

(Group  1 prompt 
appears) 

(Group  2 prompt 
appears) 


frame,  E (NL) 
e.g.  2,  E (NL) 

Q,  M,  D,  or  P (NL) 
enter  Group  1 line 

enter  Group  2 llne(a) 


- to  replace  a 
frame  (replace 
Frame  2) 


select  frame  type 


* 

ENTER  COMMAND 


& (NL) 


when  finished 


Figure  B-5.  Procedure  Guide  to  Use  of  PLANIT  Editing 
Commands  to  Modify  (Replace)  Material 


HOW  TO  RUN  A CAI  LESSON  WITH  PLANIT 
UNDER  COURSE  DEVELOPER  CONTROL 


Cut  or  Display 


(•RUN  statement 
completed 


LOGIN  OR  END 


ENTER  COMMAND 


© 


Yout  Action 

I ACCESS  PLANIT  SYSTEM  I 
(•ADD  PLANIT*PRUNS.HOT  (NL) 


key  Identifier  (NL) 
e.g.  HORN  (NL) 


^2)  [RETRIEVE  CAI  LESSONi 

CET  lessonnaae  (NL) 
e.g.  GET  FDC  (NL) 

IDENTIFY  YOURSELF*** 

author  Identifier  (NL) 
e.g.  TEAM1  (NL) 


Q 


Iexecute  lesson  I 

CLEAR  (NL) 


DONE 

e 

EX  (NL) 


(Lesson  frames  are 
administered  and  appear 
In  sequence  as  student 
would  see  then) 
when  * appears 


or 

EX  framenumber  (NL) 
e.g.,  EX  5 (NL) 
answer  (NL) 
e.g.,  CRID 
or 


4CLEAR.EX  (as  above) 

Slgu-off  PLANIT  when  finished 


Figure  B-6.  Procedure  Guide  for  Running  (Executing) 


Consents 


- load  PLANIT 

- obtain  Identifier 
from  monitor 


- use  name  assigned 
at  SAVE 


- use  Identifier 
used  at  SAVE 


- to  clear  prior 
execution  record 


- to  run  lesson  from 
the  start 


- to  run  lesson  from 
a specified  frame 

- give  right  or  wrong 
answers.  Check 
lesson  displays  and 
seqjence. 

- to  re-execute 

- See  Sign-Off 
procedure 


PLANIT  Lesson 
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APPENDIX  C 

TEC-TO-CAI  CONVERSION  WORKBOOK  EXAMPLES 

Final  TEC  Narration  Script 

Step  1 - Review  TEC  Lesson  with  Script 

Step  2 - Modify  Annotated  Script 

Step  3 - Rewrite  Script  in  Frame  Order 

Step  4 - Encode  Frames  for  CAI 
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/ 


P 


TITLE 


/i /or 

/’/-ecediy#  '/9/^y  > 


LAW  i/2  FINAL  NARRATION  SCRIPT  (typed  from  cassette) 


SON  no.  948-071-006F  M72A2  LAW.  ENGAGING  THE  TARGET DATE  22  July  1976 

,IC  TERRAIN  CONDITIONS OBJECTIVE  Identify  advantages  and 

disadvantages  of  terrain  conditions 
when  firing  the  LAW. 


TERRAIN  CONDITIONS 


1.  Another  loportant  factor  in  firing  the  LAW  la 


the  terrain. 


2.  Firing  the  LAW  without  considering  adequate 
backblast  area  can  be  hazardous. 

<*) 

3.  The  area  to  the  rear  of  the  weapon  oust  not  be 
blocked  by  a solid  obstruction. 

(*> 

It,  The  backblast  danger  area  extends  IS  meters  to 

the  rear  of  the  LAW.  Be  sure  you  are  a safe  distance 
in  front  of  any  solid  obstruction  before  you  fire. 

If  there  is  no  backblast  clearance,  do  not  fire 
the  LAW. 

(*) 

5.  Dense  undergrowth  can  also  block  backblast,  but 

to  a lesser  degree  than  a solid  obstruction.  This 

is  called  limited  backblast  clearance.  When 

practical,  select  a firing  position  where  the 

vegetation  will  obstruct  the  backblast  release. 

(*) 

1 

i • V 

' ■ 

i _ i 

Figure  C-l.  Final  TEC  Narration  Script 
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LAW  #2  FINAL  NARRATION  SCRIPT  (typed  from  cassette) 
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(•'"SON  wn  948-07 1-006F  M72A2  LAW.  ENGAGING  THE  TARGET 


DATE  22  July  1976 


TOPIC 


TERRAIN  CONDITIONS 


OBJECTIVE  Identify  advantages  and 


disadvantages  of  terrain  conditions 


when  firing  the  LAW. 


VISUAL 


AUDIO 
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1.  Another  important  factor  in  firing  the  LAW  is 
the  terrain. 

2.  Firing  the  LAW  without  considering  adequate 
backblast  area  can  be  hazardous. 

(*) 

► 

3.  The  area  to  the  rear  of  the  weapon  must  not  be 
blocked  by  a solid  obstruction. 

(*) 

4.  /The  backblast  danger  area  extends  IS  meters  to 

/ 

the  rear  of  the  LAW.  Be  sure  you  are  a safe  distance 
in  front  of  any  solid  obstruction  before  you  fire. 

If  there  is  no  backblast  clearance,  do  not  fire 
the  LAW. 

(*) 


S.  Dense  undergrowth  can  also  block  backblast,  but 

to  a lesser  degree  than  a solid  obstruction.  This 
is  called  limited  backblast  clearance.  When 


N 


practical,  select  a firing  position  where  the 

-H 

vegetation  will  obstruct  the  backblae:  me 
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Figure  C-3.  Step  2 - Modify  Annotated 
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Figure  C~4.  Step  3 - Rewrite  Script  In  Frane  Order 
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Figure  C-5.  Step  4 - Encode  Frames  for  CAI 
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APPENDIX  D 

DATA  COLLECTION  SYSTEM 

TEC/CAI  Background  Data  Questionnaire 
Daily  Individual  Activities  Log 
Monitor's  Observation  Log 
Questionnaire  (SME) 

Guidelines  for  Subject  Matter  Review 
CAI  Lesson  Review 
Questionnaire  (S) 

CAI  Student  Attitude  Questionnaire 
Student  Interaction  Record 
PLANIT  Student  Record 


D-l 


A 

r 


/ver  , 

'frecediMtS  ~ e</  j 


TEC/CAI  BACKGROUND  DATA  QUESTIONNAIRE 

DATE 

These  questions  will  help  us  to  understand  your  background  experiences  with 
respect  to  the  TEC/CAI  conversion  tasks  and  to  adjust  workshop  training  emphasis 
accordingly.  The  data  is  solely  for  use  in  the  TEC/CAI  project  and  will  not 
reveal  your  identity  when  compiled  and  described. 

NAME  RANK SSAN 

GRADE MOS JOB  TITLE 

EDUCATION  (grade  or  degree,  year,  and  major) 

RELATED  TRAINING  

ORGANIZATION  PHONE 

DATE  OF  BIRTH  LENGTH  OF  SERVICE 

TIME  IN  MOS  TIME  IN  CURRENT  JOB 

LIST  MAJOR  JOB  DUTIES 


1.  Are  you  familiar  with  the  subject  areas  for  the  TEC  materials  to  be 
converted?  (Enter  subject  areas  for  your  type  of  experience) 


Attended  classes:  Yes  No 

Taught  classes:  Yes  No 

Observed  procedures:  Yes  No 

Practice  or  combat  use:  Yes No 


2.  Have  you  written  behavioral  tasks,  training  objectives,  and  standards? 


Yes 

Subject  area(s): 

No 

Have  you  developed  Army  performance  tests?  (yes/no) 

Yes 

Printed  tests? 

Standards? 

No 

1 Live/Simulatcd  tests? 

Standards? 

Have  you  written  printed  training  support  materials? 

(e.g.,  lesson  plans) 

Yes 

Instructor  materials 

No 

Trainee  materials 

Figure  D-l.  TEC/CAI  Background  Data  Questionnaire 
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5.  Have  yuu  developed  audiovisual  training  materials?  (e.g.,  UTEC) 


Type  materials 


Updated  only 


6.  Have  you  developed  programmed  instruction  (PI)  booklets? 


Subject  areas: 


Linear? 


Branching? 


With  pictorials 


Have  you  made  use  of  training  aids  production  facilities  on  this  post? 


Graphic  arts? 
Reproduction? 


Photographic? 


Other 


8.  Are  you  familiar  with  computer-assisted  instruction  (CAI)? 

Yes  Name  of  CAI  system 

1 

No  j Have  you  written  CAI  lessons?  

Yes  CAI  language  used  

No j Subject  area(6) 

9.  Have  you  revised  training  materials  based  on  review  by  Army  subject-matter 
experts  (SMKs)? 

Yen  Subject  area(s)  


Type  materials:  AV PI 


Other 


10.  Have  you  evaluated  the  effectiveness  and  trainee  acceptance  of  training 
materials  by  collecting  trainee  data? 


ENTER  IDENTIFIERS  AND  DATE.  RECORD  HOURS  AND  DIFFICULTIES  AT  END  OF  WORKDAY. 
SUBMIT  LOG  AT  START  OF  NEXT  WORKDAY. 


WORKSHOP 

CONVERSION 

ACTIVITY 


_ during  Conversion  WORKSHOP 
Check  (✓)  as  appropriate:  durlng  CONVERSION  of  materials 


ACTIVITY  - Keep  track  of  daily  training  or  conversion  activities. 

HOURS  - Enter  time  started,  time  ended  on  daily  activities. 

DIFFICULTIES  - Concisely  describe  any  major  difficulties  by  activity. 

OTHER  CODES  - Enter  time  started,  time  ended,  for  any  activities 

done  outside  of  assigned  workday  (X)  or  for  absence  from 
assig  -»J  workday  (A  P II  S).  For  (X)  enter  ACTIVITY  codc3 
to  left  and  note  any  DIFFICULTIES. 

Figure  D-2.  Dally  Individual  Activities  Log 
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MONITOR’S  OBSERVATION  LOG 


DATE 

ACTIVITY  CODES 

Cl-Revlew  CAI  Lesson  C4-Prepare 

TP-PLANIT  on-line  training  C2-Review  TEC  Lesson  in  PLANIT 

TW-Other  workshop  training  C3-Prepare  for  Conversion  C5-Evaluate 

ENTER  ACTIVITY,  DIFFICULTIES,  ACTIONS  UNDER  "COMMENTS”. 
IDENTIFY  EQUIPMENT  USED  AS  APPROPRIATE. 


NAME 

NAME 

TIME 

COMMENTS 

TIME 

COMMENTS 

Figure  D-3.  Monitor's  Observation  Log 


QUESTIONNAIRE  (SME) 


Date: 


Grade 


SSAN 


Uhit 


MOS 


Phone 


Duty  Position 


Length  of  Military  Service 


Six  TEC  lessons  are  being  converted  into  a computer  assisted  instruction  (CAI) 
mode.  These  are: 


TEC  Lesson  No. 

1.  #949-06 1-000 1-F 

2.  #949-06 1-0002-F 

3.  #949-06 1-0003-F 

4.  #949-061-0004 

5.  #949-061-0005 

6.  #949-061-0006 


Title 

Determination  of  Direction 

Target  Location:  Polar  Plot  Method  and 
Grid  Coordinates  Method 

Locate  a Target  by  Shift  from  a known 
point 

The  Call  for  Fire 

The  Adjustment  of  Field  Artillery  Fire 
by  the  Bracketing  and  Creeping  Methods,  Part  I 

The  Adjustment  of  Field  Artillery  Fire  by  the 
Bracketing  and  Creeping  Methods,  Part  II 


1.  Which  of  the  subjects,  1 to  6 above,  are  you  currently  proficient?  Check 
(X)  which  areas. 

1.  0001  3.  0003  5.  0005  

2.  0002  4.  0004  6.  0006  


Figure  D-A.  Questionnaire  (SME) 
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2.  Which  of  the  subjects,  1 to  6 above,  have  you  been  an  instructor.  Check 
(X)  which  ones. 

1.  0001  3.  0003  5.  0005  

2.  0002 4.  0004 6.  0006  


GUIDELINES  FOR  SUBJECT  MATTER  REVIEW 


1 


1.  Is  the  task  information  and  data  technically  accurate  and  complete? 

2.  Is  the  procedure  described  for  performing  the  task  in  line  with  current 
or  upcoming  Army  doctrine  (use)? 

3.  If  Army  doctrine  does  not  prescribe  an  exact  procedure  for  performing  the 
task,  is  the  procedure  described  adequate?  Efficient?  Similar  in  basic 
respects  to  the  way  you  would  perform  it? 

4.  Are  there  any  written  statements  or  visuals  which  require  clarification — 
that  is,  the  information  looks  "right",  but  seems  hard  to  understand  or 
could  be  misinterpreted? 

5.  Is  the  language  "Army"? 

6.  Are  there  any  typographical  mistakes  in  answers  to  exercises  and  tests 
or  incorrect  alternatives  that  could  be  considered  correct. 


Please  write  comments  directly  on  the  lesson  materials  (listings  and  exhibits) 
concerning  these  points  where  this  is  practical.  Correct  answers  when  you  can. 
Thanks  for  your  help. 


Figure  D-5.  Guidelines  for  Subject  Matter  Review 
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CAI  LESSON  REVIEW 


Lesson  Name: 


Reviewer: 


A.  What  is  your  overall  inpr  ssion  of  this  lesson? 


3.  Is  the  lesson  technically  accurate  and  complete?  Yes_ 
Please  comment : 


C.  Is  the  material  in  line  with  current  or  upcoming  Army  doctrine  (use)?  Yes 


Please  comment: 


D.  Were  any  parts  of  the  lesson  particularly  good?  Yes_ 
Please  comment : 


E.  Were  any  parts  of  the  lesson  particularly  bad?  Yes 


Please 


comment : 


M-:! 


Figure  D-6.  CAI  Lesson  Review 


w 


DON’T:  CAI  Lesson  Review  "E" 


F.  Will  students  be  able  to  understand  the  language  used  in  the  lesson?  Yes_ 
No  Please  comment:  


G.  Compare  the  TEC  lesson  with  the  CAI  lesson.  (From  the  student's  view  point) 


H.  Are  the  objectives  of  the  CAI  lesson  the  same  as  the  TEC  lesson?  jes no 

If  no,  please  comment  on  how  they  are  different  


Figure  D-6.  CAI  Lesson  Review  (Cont.) 
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QUESTIONNAIRE  (S) 


Name  ____________________________________  Grade  SSAN  _________ 

Unit MOS  * 

Duty  Position GOT  Score Education 

1.  How  long  have  you  been  in  the  Army?  _______________ 

2.  Have  you  had  previous  military  service?  Yes  _________  No  ________ 

3.  When  did  you  complete  BCT  (date)  _________________________ 

4.  Are  you  currently  in  AIT  (Advanced  Individual  Training)?  Ye 3 

No 

5.  Have  you  had  Land  Navigation  (Map  Reading)?  Yes  No  ________ 

6.  Can  you  read  a military  map?  Yes  ________  No 


Figure  D-7 . Questionnaire  (S) 


CAI  STUDENT  ATTITUDE  QUESTIONNAIRE 


NAME  AND  GRADE SSAN  

UNIT  PHONE  DATE 


1.  What  did  you  think  of  the  Observed  Fire  CAI  Lessons  that  you  have  just  completed? 


2.  My  attitude  toward  CAI  material  was  that  I... 

( ) disliked  it  very  much 
( ) disliked  it 

( ) neither  liked  nor  disliked  it 
( ) liked  it 
( ) liked  it  very  much 

3.  Instructions  for  taking  the  lessons  were... 

( ) very  difficult  to  understand 
( ) difficult  to  understand 
( ) borderline 
( ) easy  to  understand 
( ) very  easy  to  understand 

I 

4.  Which  Observed  Fire  CAI  Lessons  did  you  take?  (check) 


Determination  of  Direction 

Target  Location:  Polar  Plot  Method  and  Grid  Coordinates  Method 
Locate  a Target  by  Shift  from  a Known  Point 
The  Call  for  Fire 

The  Adjustment  of  Fire  by  the  Bracketing  and  Creeping  Methods, 
The  Adjustment  of  Fire  by  the  Bracketing  and  Creeping  Methods, 
Figure  D-8.  CAI  Student  Attitude  Questionnaire 


Part  I 
Part  II 
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5.  Did  you  have  any  problems  or  difficulties  in  using  the  console  or  interacting 
with  the  computer? 


( ) Yes  ( ) No 

If  Yes  to  item  5,  please  describe  your  most  serious  problem  or  difficulty. 


6.  The  lessons  covered  the  areas  listed  in  (4)  above.  Were  any  of  these,  or 
parts  of  these,  particularly  good,  and  tell  why. 


7.  Were  any  of  the  lessons  particularly  bad  and  tell  why. 


Figure  D-8.  CAI  Student  Attitude  Questionnaire  (Cont.) 
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6.  I think  that  this  CAI  method  of  instruction/learning  la... 

( ) very  effective 
( ) effective 
( ) borderline 
( ) ineffective 
( ) very  ineffective 

9.  For  satisfactory  understanding  of  the  subject  being  studied,  the  amount 
of  time  provided  was: 

( ) much  too  long 
( ) fairly  long 
( ) about  right 
( ) fairly  short 
( ) much  too  short 

10.  For  satisfactory  understanding  of  the  subject  being  studied,  the  amount  of 
material  (information)  provided  was: 

( ) much  too  large 
( ) fairly  large 
( ) about  right 
( ) fairly  small 
( ) much  too  small 

11.  The  technical  detail  provided  was: 

( ) very  satisfactory 
( ) satisfactory 
( ) borderline 
( ) unsatisfactory 
( ) very  unsatisfactory 


Figure  D-8.  CAI  Student  Attitude  Questionnaire  (Cont.) 


12. 


The  organization  of  the  material  presented  was: 


( ) very  satisfactory 
( ) satisfactory 
( ) borderline 
( ) unsatisfactory 
( ) very  unsatisfactory 

13.  My  understanding  of  the  material  presented  was: 

( ) very  satisfactory 
( ) satisfactory 
( ) borderline 
( ) unsatisfactory 
( ) very  unsatisfactory 

14.  The  quantity  of  the  off-line  course  exhibits  provided  was: 

( ) very  satisfactory 
( ) satisfactory 
( ) borderline 
( ) unsatisfactory 
( ) very  unsatisfactory 

15.  Were  any  of  the  exhibits  inaccurate? 

( ) Yes  ( ) No 

If  Yes,  please  describe:  


16.  Were  any  of  the  exhibits  irrelevant  or  unnecessary? 
( ) Yes  ( ) No  If  Yes,  which?  


Figure  D-8.  CAI  Student  Attitude  Questionnaire  (Cont.) 

D-16 


17.  Can  you  think  of  any  other  exhibits  that  should  be  added  to  the  set? 


( ) Yes 


( ) No  If  Yes,  please  describe: 


18.  If  you  had  to  carry  out  the  tasks  on  observed  fire  covered  in  the  course, 
how  well  could  you  do  them? 

( ) very  effectively 
( ) effectively 
( ) borderline 
( ) ineffectively 
( ) very  ineffectively 

19.  Have  you  ever  had  this  type  (CA1)  training  before? 

( ) Yes  ( ) No 

20.  Does  this  type  of  training  make  Army  instruction  better? 

( ) Yes  ( ) No 

Why? 


21.  Is  this  type  of  training  interesting  to  you? 
( ) Yes  ( ) No  ( ) Not  Sure 
Why? 


Figure  D-8.  CAi  student  Attitude  Questionnaire  (Cont.) 
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22. 


Does  this  type  of  training  make  It  easy  for  you  to  learn? 
( ) Yes  ( ) No  ( ) Don't  Know 

Why?  


23.  Do  you  like  this  type  of  training* 

( ) Yes  ( ) No  ( ) Undecided 

Why? 


1 in 
« 

Figure  D-8.  CAI  Student  Attitude  Questionnaire  (Cont.) 


D-18 


” 


♦ • T WG£ r lGCRTIOM*# 


THE  OBJECTIVE  Of  THIS  TEST  IS  TO  EVlAUATE  YOUR  ABILITY  TOl 


1.  LOCATE  A TARGET  BY  POLAR  PLOT  WHEN  THE  TARGE T-REFERENCE  POINT 
IS  CMOWH 

£.  LOCATE  a TARGET  BY  POLAR  PLOT  BY  ESTIMATING  BATA  fROH  A HAR 

3.  LOCATE  A TARGET  BY  GRIB  COORBIhATE*  BY  £'.  TIHATIMG  BATA 
PROfl  A MAP 


».TYP£  '60  TO  CONTINUE) 


♦ 

GO 


SUCCESSFUL  COMPLETION  of  this  test  INBICATES  THAT  YOU  AlREABY 
POSSESS  THE  NECESSARY  SKILLS  TO  LOCATE  TARGETS  BY  THE  POlAR 

plot  methob  anb  the  grib  coorbinate  methob.anb  will  not  have 

TO  take  THE  LESSON  ON  •• TARGET  LOCATION 

BO  YOU  HAVE  A PENCIL.  PAPER,  ANB  THE  TARGET  LOCATION  TEST  HANBOU T '< 
IF  NOT.  GET  THEN  FROH  THE  MONITOR.  WHEN  YOU  APE  REPAY, 


< TYPE  GO'  TO  CONTINUE  > 


GO 

THE  INFORNAT ION  TO  LOCATE  A TARGET  BY  THE  POlAR  PLOT  METHOB  IS 
SHOWN  IN  FIGURE  1.  OF  THE  HANBOUT.  BIRECTION  TO  THE  KNOWN  REFERENCE 
POINT  yHIllTOR  610)  IS  315  BEGREES.  THE  ESTIMATEB  ANGLE  FROM 
HILLTOP  610  TO  THE  TARGET  IS  10  BEGREES. ‘THE  ESTIMATEB  BISTANCE 
TO  THE  TARGET  IS  3000  METERS.  ANB  THE  VERTICAL  SHIFT  TO  THE 
TARGET  IS  «-I0  METEPS. 

STUBY  FIGURE  I. 

Y TYPE  ' GO  TO  CONTINUE) 


•30 


THE  RELATIVE  LOCATION  OF  THE  OBSERVER .REFERENCE  POINT  ANB  TARGET. 

along  with  the  known  infornation.  is  shown  in  figure  i.  when 

LOCATING  THIS  TARGET  BY  THE  POLAR  PLOT  METHOB.  WHAT  TWO  ElEHENTS 
OF  TARGET  LOCATION  NUST  BE  SENT  TO  THE  PIPING  UNIT?  TYPE  THE 
^ETTER  CORRESPONBING  TO  YOUR  ANSWER.  (A.B.C.  OR  B) 


A.  BISTANCE  ANB  RANGE 

B.  B I PEC T I ON  ANB  BISTANCE 

C.  BISTANCE  ANB  TARGET  TYPE 
b.  elevation  anb  bistance 


Figure  D-9.  Student  Interaction  Record 
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SUflHfiRY-  ONLY 
• N 


N 

DATE  O'  0 ' 1 

LESSON  NAME  BCB > STUDENT  NAME 


jTRAME 

START 

TYPE 

TIME 

MIN.- 

SEC 

NEUTRAL 

RIGHT-* 

WRONG- 

1 .00 

Q 

63.59 

0 

2.00 

0 

63.59 

0 

3.00 

0 

63.60 

0 

4.00 

0 

63.61 

0 

S.00 

0 

63.63 

0 

6.00 

M 

63.60 

♦ 

7.00 

0 

63 .63 

0 

8.00 

M 

63.63 

* 

9.00 

D 

63 .63 

0 

10.00 

0 

63.62 

0 

11.00 

a 

63.58 

0 

12.00 

M 

63.61 

♦ 

13.00 

0 

63.60 

0 

14.00 

M 

63.62 

♦ 

15.00 

0 

63.63 

0 

16.00 

0 

63.60 

0 

17.00 

Q 

63.61 

0 

18.00 

0 

63.63 

0 

19.00 

M 

63.60 

♦ 

20.00 

M 

63.58 

♦ 

21.00 

M 

63.62 

♦ 

22.00 

M 

63.61 

♦ 

23.00 

M 

63.61 

•f 

24.00 

P 

63.63 

0 

25.00 

0 

63.58 

0 

25.50 

0 

63.61 

0 

26.00 

Q 

63.63 

0 

27.00 

0 

63.61 

0 

28.00 

M 

63.63 

4 

29.00 

M 

63.59 

♦ 

30.00 

M 

63.60 

♦ 

31  .00 

M 

63.62 

♦ 

32.00 

M 

63.60 

♦ 

33.00 

M 

63.60 

♦ 

34.00 

D 

63.63 

0 

35.00 

0 

63.59 

0 

36.00 

0 

63.59 

0 

37..  00 

e 

63.63 

0 

39  .>00 

0 

63.59 

0 

40.00 

0 

63.62 

0 

41.00  M 

NUMBER  RIGHT 

63.59 

16 

♦ 

NUMBER  WRONG  0 


NUMBER  ENTRIES  41 
NUMBER  TIME-OUTS  0 
TOTAL  TIME  2624.37 


STUDENT  HISTORY 

ANSWER  LABEL 
TAG 

O'O'l  0.00 

A BCFPRE 

A 

A 

A 

A 

B 

A 

A 

A 

A 

C 

A 

B 

A 

A 

A 

A 

B 

D 

B 

B 

C 

A . 

A 

A 

A 

C 

A 

C 

A 

A 

D 

A 

A 

A 

A 

A 

B 


Figure  0-10.  PLANIT  Student  Record 
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APPENDIX  E 


i 


TEC-TO-CAI  CONVERSION  WORKSHOP  SCHEDULE 


CONVERSION  WORKSHOP  SCHEDULE 

BLOCK  1 (WEEK  1):  PREPARE  TEC  MATERIAL  FOR  CAI  CONVERSION 


" ^ 


Figure  E-l . Conversion  Workshop  Schedule 

Block  1 (Week  1):  Prepare  TEC 
Material  for  CAI  Conversion 


r 1 


Figure  E-3.  Conversion  Workshop  Schedule 

Block  3 (Week  3) : Evaluate  and 
Revise  Converted  Materials 


